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1 Introduction 


Meetings. We meet twice a week, on Tuesdays and 
Thursdays, from 1:15 to 2:30pm, in room D106 LSRC. 


Communication. The course material will be delivered 
in the two weekly lectures. A written record of the lec- 
tures will be available on the web, usually a day after the 
lecture. The web also contains other information, such as 
homework assignments, solutions, useful links, etc. The 
main supporting text is 


TARJAN. Data Structures and Network Algorithms. SIAM, 
1983. 


The book focuses on fundamental data structures and 
graph algorithms, and additional topics covered in the 
course can be found in the lecture notes or other texts in 
algorithms such as 


KLEINBERG AND TARDOS. Algorithm Design. Pearson Ed- 
ucation, 2006. 


Examinations. There will be a final exam (covering the 
material of the entire semester) and a midterm (at the be- 
ginning of October), You may want to freshen up your 
math skills before going into this course. The weighting 
of exams and homework used to determine your grades is 


homework 35%, 
midterm 25%, 
final 40%. 


Homework. We have seven homeworks scheduled 
throughout this semester, one per main topic covered in 
the course. The solutions to each homework are due one 
and a half weeks after the assignment. More precisely, 
they are due at the beginning of the third lecture after the 
assignment. The seventh homework may help you prepare 
for the final exam and solutions will not be collected. 


Rule 1. The solution to any one homework question must 
fit on a single page (together with the statement of the 
problem). 


Rule 2. The discussion of questions and solutions before 
the due date is not discouraged, but you must formu- 
late your own solution. 


Rule 3. The deadline for turning in solutions is 10 min- 
utes after the beginning of the lecture on the due date. 


Overview. The main topics to be covered in this course 
are 


I Design Techniques; 

II Searching; 

MI Prioritizing; 

IV Graph Algorithms; 

V Topological Algorithms; 
VI Geometric Algorithms; 
VII NP-completeness. 


The emphasis will be on algorithm design and on algo- 
rithm analysis. For the analysis, we frequently need ba- 
sic mathematical tools. Think of analysis as the measure- 
ment of the quality of your design. Just like you use your 
sense of taste to check your cooking, you should get into 
the habit of using algorithm analysis to justify design de- 
cisions when you write an algorithm or a computer pro- 
gram. This is a necessary step to reach the next level in 
mastering the art of programming. I encourage you to im- 
plement new algorithms and to compare the experimental 
performance of your program with the theoretical predic- 
tion gained through analysis. 


I DESIGN TECHNIQUES 


aAbBwWh 


Divide-and-Conquer 
Prune-and-Search 

Dynamic Programming 
Greedy Algorithms 

First Homework Assignment 


2 Divide-and-Conquer 


We use quicksort as an example for an algorithm that fol- 
lows the divide-and-conquer paradigm. It has the repu- 
tation of being the fasted comparison-based sorting algo- 
rithm. Indeed it is very fast on the average but can be slow 
for some input, unless precautions are taken. 


The algorithm. Quicksort follows the general paradigm 
of divide-and-conquer, which means it divides the un- 
sorted array into two, it recurses on the two pieces, and it 
finally combines the two sorted pieces to obtain the sorted 
array. An interesting feature of quicksort is that the divide 
step separates small from large items. As a consequence, 
combining the sorted pieces happens automatically with- 
out doing anything extra. 


void QUICKSoRT(int £, r) 
if L< r then m = SPLIT(¢,r); 
QUICKSORT(4, m — 1); 
QUICKSORT(m + 1, r) 
endif. 


We assume the items are stored in A[0..n — 1]. The array 
is sorted by calling QUICKSORT(0,n — 1). 


Splitting. The performance of quicksort depends heav- 
ily on the performance of the split operation. The effect of 
splitting from £ to r is: 


e x = A[é] is moved to its correct location at Afm]; 
e noitemin A[é..m — 1] is larger than x; 


e noitem in Alm + 1..r] is smaller than z. 


Figure 1 illustrates the process with an example. The nine 
items are split by moving a pointer ¿ from left to right 
and another pointer j from right to left. The process stops 
when 2 and j cross. To get splitting right is a bit delicate, 
in particular in special cases. Make sure the algorithm is 
correct for (i) x is smallest item, (ii) x is largest item, (iii) 
all items are the same. 


int SPLIT(int £,r) 
x = Al; i=f j=r+1; 
repeat repeat i++ until wz < Ali]; 
repeat j-- until «> A[j]; 
ifi<jthen SWAP(i,j) endif 
untili> j; 
SwaP(l,7); return j. 
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Figure 1: First, ¿ and j stop at items 9 and 1, which are then 
swapped. Second, 7 and j cross and the pivot, 7, is swapped with 
item 2. 


Special cases (i) and (iii) are ok but case (ii) requires a 
stopper at A[r + 1]. This stopper must be an item at least 
as large as x. If r < n — 1 this stopper is automatically 
given. For r = n — 1, we create such a stopper by setting 
Ajn] = +œ. 


Running time. The actions taken by quicksort can be 
expressed using a binary tree: each (internal) node repre- 
sents a call and displays the length of the subarray; see 
Figure 2. The worst case occurs when A is already sorted. 


Figure 2: The total amount of time is proportional to the sum 
of lengths, which are the numbers of nodes in the corresponding 
subtrees. In the displayed case this sum is 29. 


In this case the tree degenerates to a list without branch- 
ing. The sum of lengths can be described by the following 
recurrence relation: 


n nt 1 
T = T(n-1) = . = i 
m) = meray = Yi = (73) 

The running time in the worst case is therefore in O(n”). 


In the best case the tree is completely balanced and the 
sum of lengths is described by the recurrence relation 


T(n) = near (44), 


If we assume n = 2” — 1 we can rewrite the relation as 


U(k) = (2*-1)+2-U(k—-1) 
= (2*-—1)+4+2(a"*—1)+...+2"-1(2-1) 
k-1 
k-2k—S° 2! 
= 2H. k (Æ — 1) 


= (n+1)-log,(n+1)—n. 


The running time in the best case is therefore in 
O(n logn). 


Randomization. One of the drawbacks of quicksort, as 
described until now, is that it is slow on rather common 
almost sorted sequences. The reason are pivots that tend 
to create unbalanced splittings. Such pivots tend to oc- 
cur in practice more often than one might expect. Hu- 
man and often also machine generated data is frequently 
biased towards certain distributions (in this case, permuta- 
tions), and it has been said that 80% of the time or more, 
sorting is done on either already sorted or almost sorted 
files. Such situations can often be helped by transferring 
the algorithm’s dependence on the input data to internally 
made random choices. In this particular case, we use ran- 
domization to make the choice of the pivot independent of 
the input data. Assume RANDOM(¢,7r) returns an integer 
p € |£, r] with uniform probability: 


1 


Prob[RANDOM(f,r) =p] = ae 
r= 


for each 4 < p < r. In other words, each p € [£,r] is 
equally likely. The following algorithm splits the array 
with a random pivot: 


int RSPLIT(int 4,r) 
p = RANDOM(£, r); SWAP(£, p); 
return SPLIT(/, r). 


We get a randomized implementation by substituting 
RSPLIT for SPLIT. The behavior of this version of quick- 
sort depends on p, which is produced by a random number 
generator. 


Average analysis. We assume that the items in A[0..n — 
1] are pairwise different. The pivot splits A into 


A[0..m—1], Alm], Afm + 1..n — 1]. 


By assumption on function RSPLIT, the probability for 
each m € [0,n — 1] is 4. Therefore the average sum 
of array lengths split by QUICKSORT is 


T(n) = nt 2-7 (Tm) +T- m- 1). 
m=0 


To simplify, we multiply with n and obtain a second rela- 
tion by substituting n — 1 for n: 


n—1 


n-T(n) = n?+2-5°T(i), (1) 
7=0 


n—2 


(n—1)-T(n-1) = (n-1)?+2-S°7(i). 2) 
i=0 


Next we subtract (2) from (1), we divide by n(n + 1), we 
use repeated substitution to express T(n) as a sum, and 
finally split the sum in two: 


T(n) — T(n-1) 2n—1 

n+l n “ n(n +1) 
_ T(n-2) 2n — 3 2n-—1 
~ n=1 | (n—=1)n nnt!) 


n 


1 1 
F 2 È 


Bounding the sums. The second sum is solved directly 
by transformation to a telescoping series: 


Saag au. 


The first sum is bounded from above by the integral of + 
for x ranging from | to n + 1; see Figure 3. The sum 
of = is the sum of areas of the shaded rectangles, and 
because all rectangles lie below the graph of 4 we get a 
bound for the total rectangle area: 
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Figure 3: The areas of the rectangles are the terms in the sum, 
and the total rectangle area is bounded by the integral from 1 
through n + 1. 


We plug this bound back into the expression for the aver- 
age running time: 


T(n) < (n+) 
< Di td) «inte 


= = (n + 1) -logy(n + 1). 
In words, the running time of quicksort in the average case 
is only a factor of about 2/ log, e = 1.386... slower than 
in the best case. This also implies that the worst case can- 
not happen very often, for else the average performance 
would be slower. 


Stack size. Another drawback of quicksort is the recur- 
sion stack, which can reach a size of Q(n) entries. This 
can be improved by always first sorting the smaller side 
and simultaneously removing the tail-recursion: 


void QUICKSoRT(int £, r) 
i=l; 9 =7; 
whilez<jdo 
m = RSPLIT(i, j); 
ifm—-it<g-m 
then QUICKSORT(i,m—1);i=m+1 
else QUICKSORT(m + 1,7); j =m—1 
endif 
endwhile. 


In each recursive call to QUICKSORT, the length of the ar- 
ray is at most half the length of the array in the preceding 
call. This implies that at any moment of time the stack 
contains no more than 1 + logs n entries. Note that with- 
out removal of the tail-recursion, the stack can reach Q(n) 
entries even if the smaller side is sorted first. 


Summary. Quicksort incorporates two design tech- 
niques to efficiently sort n numbers: divide-and-conquer 
for reducing large to small problems and randomization 
for avoiding the sensitivity to worst-case inputs. The av- 
erage running time of quicksort is in O(n logn) and the 
extra amount of memory it requires is in O(logn). For 
the deterministic version, the average is over all n! per- 
mutations of the input items. For the randomized version 
the average is the expected running time for every input 
sequence. 


3  Prune-and-Search 


We use two algorithms for selection as examples for the 
prune-and-search paradigm. The problem is to find the 
i-smallest item in an unsorted collection of n items. We 
could first sort the list and then return the item in the 2-th 
position, but just finding the i-th item can be done faster 
than sorting the entire list. As a warm-up exercise consider 
selecting the 1-st or smallest item in the unsorted array 
A{l..n]. 


min = 1; 
for j =2to ndo 

if A[j] < A[min] then min = j endif 
endfor. 


The index of the smallest item is found in n — 1 com- 
parisons, which is optimal. Indeed, there is an adversary 
argument, that is, with fewer than n — 1 comparisons we 
can change the minimum without changing the outcomes 
of the comparisons. 


Randomized selection. We return to finding the ʻi- 
smallest item for a fixed but arbitrary integer 1 < i < n, 
which we call the rank of that item. We can use the split- 
ting function of quicksort also for selection. As in quick- 
sort, we choose a random pivot and split the array, but we 
recurse only for one of the two sides. We invoke the func- 
tion with the range of indices of the current subarray and 
the rank of the desired item, 7. Initially, the range consists 
of all indices between £ = 1 and r = n, limits included. 


int RSELECT(int @,7,7) 
q = RSPLIT(é,r); m=q—+1; 
if i< m then return RSELECT(E, q — 1,7?) 
elseifi =m then returnq 
else return RSELECT(q + 1,r,i— m) 
endif. 


For small sets, the algorithm is relatively ineffective and 
its running time can be improved by switching over to 
sorting when the size drops below some constant thresh- 
old. On the other hand, each recursive step makes some 
progress so that termination is guaranteed even without 
special treatment of small sets. 


Expected running time. For each 1 < m < n, the 
probability that the array is split into subarrays of sizes 
m -— 1l and n— mis +. For convenience we assume that n 


n 


is even. The expected running time increases with increas- 
ing number of items, T (k) < T(m) if k < m. Hence, 


1 n 
T(n) < n+ — > max{T(m- 1), T(n- m)} 
n 
m=1 
2 n 
< n+- T(m- 1) 
n 
m=%+1 


Assume inductively that T(m) < cm for m < n and 
a sufficiently large positive constant c. Such a constant 
c can certainly be found for m = 1, since for that case 
the running time of the algorithm is only a constant. This 
establishes the basis of the induction. The case of n items 
reduces to cases of m < n items for which we can use the 
induction hypothesis. We thus get 


2 
T(n) < n+ — 5 m—1 
m= F+1 
= nt+c (n-1)-$- (441) 
_ 3c 3c 
er 2 


Assuming c > 4 we thus have T(n) < cn as required. 
Note that we just proved that the expected running time of 
RSELECT is only a small constant times that of RSPLIT. 
More precisely, that constant factor is no larger than four. 


Deterministic selection. The randomized selection al- 
gorithm takes time proportional to n? in the worst case, 
for example if each split is as unbalanced as possible. It is 
however possible to select in O(n) time even in the worst 
case. The median of the set plays a special role in this al- 


gorithm. It is defined as the z-smallest item where i = r+ 
if n is odd and ¿ = 5 or r2 if n is even. The determinis- 


tic algorithm takes five steps to select: 


Step 1. Partition the n items into [| groups of size 
at most 5 each. 


Step 2. Find the median in each group. 
Step 3. Find the median of the medians recursively. 


Step 4. Split the array using the median of the medians 
as the pivot. 


Step 5. Recurse on one side of the pivot. 


It is convenient to define k = [2] and to partition such 


that each group consists of items that are multiples of k 
positions apart. This is what is shown in Figure 4 provided 


we arrange the items row by row in the array. 
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Figure 4: The 43 items are partitioned into seven groups of 5 and 
two groups of 4, all drawn vertically. The shaded items are the 
medians and the dark shaded item is the median of medians. 


Implementation with insertion sort. We use insertion 
sort on each group to determine the medians. Specifically, 
we sort the items in positions £, + k, €+2k,@+3k, @+4k 
of array A, for each £. 


void ISorT(int 4, k,n) 

j=l+k; 

while j <ndot=J; 
while i> land Ali] > Aļi — k] do 

SwaP(i,i— k); i=i—k 

endwhile; 
j=j+k 

endwhile. 


Although insertion sort takes quadratic time in the worst 
case, it is very fast for small arrays, as in this applica- 
tion. We can now combine the various pieces and write 
the selection algorithm in pseudo-code. Starting with the 
code for the randomized algorithm, we first remove the 
randomization and second add code for Steps 1, 2, and 3. 
Recall that ¿ is the rank of the desired item in A[é..r]. Af- 
ter sorting the groups, we have their medians arranged in 
the middle fifth of the array, A[@+ 2k..¢+ 3k— 1], and we 
compute the median of the medians by recursive applica- 
tion of the function. 


int SELECT(int £, r,i) 

k= [(r— £+ 1)/5]; 

for j = 0 to k — 1 do ISORT(l + j, k,r) endfor; 

m’ = SELECT(é + 2k, l + 3k — 1, | (k + 1)/2]); 

SwaP(é,m’); q = SPLIT(¢,r); m =q— £+ 1; 

if i< m then return SELECT(l,q— 1,i) 
elseifi =m then return q 
else return SELECT(q + 1,r,i — m) 

endif. 


Observe that the algorithm makes progress as long as there 
are at least three items in the set, but we need special treat- 
ment of the cases of one or of two items. The role of the 
median of medians is to prevent an unbalanced split of 


the array so we can safely use the deterministic version of 
splitting. 


Worst-case running time. To simplify the analysis, we 
assume that n is a multiple of 5 and ignore ceiling and 
floor functions. We begin by arguing that the number of 
items less than or equal to the median of medians is at least 
z, These are the first three items in the sets with medians 
less than or equal to the median of medians. In Figure 4, 
these items are highlighted by the box to the left and above 
but containing the median of medians. Symmetrically, the 
number of items greater than or equal to the median of 
medians is at least on The first recursion works on a set 
of 5 medians, and the second recursion works on a set of 
at most aa items. We have 


T(n) < n+r(3)+r (F). 


We prove T(n) = O(n) by induction assuming T(m) < 
c- m form < n and c a large enough constant. 


T(n) < ntoentoon 


Assuming c > 10 we have T(n) < cn, as required. Again 
the running time is at most some constant times that of 
splitting the array. The constant is about two and a half 
times the one for the randomized selection algorithm. 


A somewhat subtle issue is the presence of equal items 
in the input collection. Such occurrences make the func- 
tion SPLIT unpredictable since they could occur on either 
side of the pivot. An easy way out of the dilemma is to 
make sure that the items that are equal to the pivot are 
treated as if they were smaller than the pivot if they occur 
in the first half of the array and they are treated as if they 
were larger than the pivot if they occur in the second half 
of the array. 


Summary. The idea of prune-and-search is very similar 
to divide-and-conquer, which is perhaps the reason why 
some textbooks make no distinction between the two. The 
characteristic feature of prune-and-search is that the recur- 
sion covers only a constant fraction of the input set. As we 
have seen in the analysis, this difference implies a better 
running time. 


It is interesting to compare the randomized with the de- 
terministic version of selection: 


e the use of randomization leads to a simpler algorithm 
but it requires a source of randomness; 


e upon repeating the algorithm for the same data, the 
deterministic version goes through the exact same 
steps while the randomized version does not; 


e we analyze the worst-case running time of the deter- 
ministic version and the expected running time (for 
the worst-case input) of the randomized version. 


All three differences are fairly universal and apply to other 
algorithms for which we have the choice between a deter- 
ministic and a randomized implementation. 
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4 Dynamic Programming 


Sometimes, divide-and-conquer leads to overlapping sub- 
problems and thus to redundant computations. It is not 
uncommon that the redundancies accumulate and cause 
an exponential amount of wasted time. We can avoid 
the waste using a simple idea: solve each subproblem 
only once. To be able to do that, we have to add a cer- 
tain amount of book-keeping to remember subproblems 
we have already solved. The technical name for this de- 
sign paradigm is dynamic programming. 


Edit distance. We illustrate dynamic programming us- 
ing the edit distance problem, which is motivated by ques- 
tions in genetics. We assume a finite set of characters 
or letters, X, which we refer to as the alphabet, and we 
consider strings or words formed by concatenating finitely 
many characters from the alphabet. The edit distance be- 
tween two words is the minimum number of letter inser- 
tions, letter deletions, and letter substitutions required to 
transform one word to the other. For example, the edit 
distance between FOOD and MONEY is at most four: 


FOOD — MOOD — MOND — MONED — MONEY 


A better way to display the editing process is the gap rep- 
resentation that places the words one above the other, with 
a gap in the first word for every insertion and a gap in the 
second word for every deletion: 

F O O D 

MON EY 
Columns with two different characters correspond to sub- 
stitutions. The number of editing steps is therefore the 
number of columns that do not contain the same character 
twice. 


Prefix property. It is not difficult to see that you cannot 
get from FOOD to MONEY in less than four steps. However, 
for longer examples it seems considerably more difficult 
to find the minimum number of steps or to recognize an 
optimal edit sequence. Consider for example 


A L GOR I T HM 
A L T RUIS T 1 C 


Is this optimal or, equivalently, is the edit distance between 
ALGORITHM and ALTRUISTIC six? Instead of answering 
this specific question, we develop a dynamic program- 
ming algorithm that computes the edit distance between 
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an m-character string A[1..m] and an n-character string 
B{1..n]. Let E(i, j) be the edit distance between the pre- 
fixes of length i and j, that is, between A[1..i] and B[1..ġ]. 
The edit distance between the complete strings is therefore 
E(m,n). A crucial step towards the development of this 
algorithm is the following observation about the gap rep- 
resentation of an optimal edit sequence. 


PREFIX PROPERTY. If we remove the last column of an 
optimal edit sequence then the remaining columns 
represent an optimal edit sequence for the remaining 
substrings. 


We can easily prove this claim by contradiction: if the 
substrings had a shorter edit sequence, we could just glue 
the last column back on and get a shorter edit sequence for 
the original strings. 


Recursive formulation. We use the Prefix Property to 
develop a recurrence relation for Æ. The dynamic pro- 
gramming algorithm will be a straightforward implemen- 
tation of that relation. There are a couple of obvious base 
cases: 


e Erasing: we need 7 deletions to erase an 7-character 
string, E(i, 0) = i. 

e Creating: we need j insertions to create a j- 
character string, Æ(0, j) = j. 


In general, there are four possibilities for the last column 
in an optimal edit sequence. 


e Insertion: the last entry in the top row is empty, 
E(i,j) = Eli j- 1)+1. 

e Deletion: the last entry in the bottom row is empty, 

e Substitution: both rows have characters in the last 
column that are different, E(i, j) = E(i — 1,j — 
1)+1. 

e No action: both rows end in the same character, 
B(i,j) = E(i-1,j— 1). 


Let P be the logical proposition A[i] A B[j] and denote 
by | P| its indicator variable: |P| = 1 if P is true and |P| = 
0 if P is false. We can now summarize and for 7,7 > 0 
get the edit distance as the smallest of the possibilities: 


BG,f=-1)+1 
E(i — 1.9) +1 
mi-i- Deit 


min 


The algorithm. If we turned this recurrence relation di- 
rectly into a divide-and-conquer algorithm, we would have 
the following recurrence for the running time: 

T(m,n) = T(m,n—1)+T(m-—1,n) 
+T(m—1,n—1)4+1. 


The solution to this recurrence is exponential in m and n, 
which is clearly not the way to go. Instead, let us build 
an m + 1 times n + 1 table of possible values of E(i, j). 
We can start by filling in the base cases, the entries in the 
0-th row and column. To fill in any other entry, we need 
to know the values directly to the left, directly above, and 
both to the left and above. If we fill the table from top to 
bottom and from left to right then whenever we reach an 
entry, the entries it depends on are already available. 


int EDITDISTANCE(int m,n) 
for i = 0 to m do Eli,0| =i endfor; 
for j = 1 to n do E(0,j] = j endfor; 
fori=ltomdo 
forj=l1tondo 
Eli, j] = min{ Eļi, j — 1] +1, Efi - 1,7] +1, 
Ei- 1,j- 1] + |A] # BUI 
endfor 
endfor; 
return E[m, n]. 


Since there are (m+1)(n+1) entries in the table and each 
takes a constant time to compute, the total running time is 
in O(mn). 


An example. The table constructed for the conversion of 
ALGORITHM to ALTRUISTIC is shown in Figure 5. Boxed 
numbers indicate places where the two strings have equal 
characters. The arrows indicate the predecessors that de- 
fine the entries. Each direction of arrow corresponds to a 
different edit operation: horizontal for insertion, vertical 
for deletion, and diagonal for substitution. Dotted diago- 
nal arrows indicate free substitutions of a letter for itself. 


Recovering the edit sequence. By construction, there 
is at least one path from the upper left to the lower right 
corner, but often there will be several. Each such path 
describes an optimal edit sequence. For the example at 
hand, we have three optimal edit sequences: 
A L GORI T H M 
A L T R UIs TIC 


12 


0 132335455 56 578 9 >10 
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Figure 5: The table of edit distances between all prefixes of 
ALGORITHM and of ALTRUISTIC. The shaded area highlights the 
optimal edit sequences, which are paths from the upper left to 
the lower right corner. 


A L GOR I T HM 
A L T RUISTIC 
A L GOR I T HM 
A L T R UIs TIC 


They are easily recovered by tracing the paths backward, 
from the end to the beginning. The following algorithm 
recovers an optimal solution that also minimizes the num- 
ber of insertions and deletions. We call it with the lengths 
of the strings as arguments, R (m, n). 


void R(int i, j) 
ifi>Oorj>Othen 
switch incoming arrow: 
case N: R(i-—1,j — 1); print(Ali], B[j]) 
case |: R(i-1,/7); print(Ali, -) 
case >: R(i,j — 1); print(_, B[j]). 
endswitch 
endif. 


Summary. The structure of dynamic programming is 
again similar to divide-and-conquer, except that the sub- 
problems to be solved overlap. As a consequence, we get 
different recursive paths to the same subproblems. To de- 
velop a dynamic programming algorithm that avoids re- 
dundant solutions, we generally proceed in two steps: 


1. We formulate the problem recursively. In other 
words, we write down the answer to the whole prob- 
lem as a combination of the answers to smaller sub- 
problems. 


2. We build solutions from bottom up. Starting with the 
base cases, we work our way up to the final solution 
and (usually) store intermediate solutions in a table. 


For dynamic programming to be effective, we need a 
structure that leads to at most some polynomial number 
of different subproblems. Most commonly, we deal with 
sequences, which have linearly many prefixes and suffixes 
and quadratically many contiguous substrings. 
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5 Greedy Algorithms 


The philosophy of being greedy is shortsightedness. Al- 
ways go for the seemingly best next thing, always op- 
timize the presence, without any regard for the future, 
and never change your mind about the past. The greedy 
paradigm is typically applied to optimization problems. In 
this section, we first consider a scheduling problem and 
second the construction of optimal codes. 


A scheduling problem. Consider a set of activities 
1,2,...,n. Activity 7 starts at time s; and finishes 
at time f; > si. Two activities ¿ and j overlap if 
[si, fi] A [s;, fj] # O. The objective is to select a maxi- 
mum number of pairwise non-overlapping activities. An 
example is shown in Figure 6. The largest number of ac- 


—>— 
time 


Figure 6: A best schedule is c, e, f, but there are also others of 
the same size. 


tivities can be scheduled by choosing activities with early 
finish times first. We first sort and reindex such that i < 7 
implies fi < fj. 


S = {1}; last = 1; 
fori =2 tondo 
if flast < si then 
S = SU {i}; last =i 
endif 
endfor. 


The running time is O(n log n) for sorting plus O(n) for 
the greedy collection of activities. 


It is often difficult to determine how close to the opti- 
mum the solutions found by a greedy algorithm really are. 
However, for the above scheduling problem the greedy 
algorithm always finds an optimum. For the proof let 
1 = îi < ig < ... < ip be the greedy schedule con- 
structed by the algorithm. Let jı < jg < ... < je be any 
other feasible schedule. Since 7; = 1 has the earliest finish 
time of any activity, we have f;, < f;,. We can therefore 
add 2; to the feasible schedule and remove at most one ac- 
tivity, namely jı. Among the activities that do not overlap 
71, i2 has the earliest finish time, hence fi, < fja. We can 
again add 72 to the feasible schedule and remove at most 
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one activity, namely jə (or possibly jı if it was not re- 
moved before). Eventually, we replace the entire feasible 
schedule by the greedy schedule without decreasing the 
number of activities. Since we could have started with a 
maximum feasible schedule, we conclude that the greedy 
schedule is also maximum. 


Binary codes. Next we consider the problem of encod- 
ing a text using a string of Os and 1s. A binary code maps 
each letter in the alphabet of the text to a unique string 
of Os and 1s. Suppose for example that the letter ‘t’ is 
encoded as ‘001’, ‘h’ is encoded as ‘101’, and ‘e’ is en- 
coded as ‘01’. Then the word ‘the’ would be encoded as 
the concatenation of codewords: ‘00110101’. This partic- 
ular encoding is unambiguous because the code is prefix- 
free: no codeword is prefix of another codeword. There is 


t h 


Figure 7: Letters correspond to leaves and codewords correspond 
to maximal paths. A left edge is read as ‘0’ and a right edge as 
‘1’. The tree to the right is full and improves the code. 


a one-to-one correspondence between prefix-free binary 
codes and binary trees where each leaf is a letter and the 
corresponding codeword is the path from the root to that 
leaf. Figure 7 illustrates the correspondence for the above 
3-letter code. Being prefix-free corresponds to leaves not 
having children. The tree in Figure 7 is not full because 
three of its internal nodes have only one child. This is an 
indication of waste. The code can be improved by replac- 
ing each node with one child by its child. This changes 
the above code to ‘00’ for ‘t’, ‘1’ for ‘h’, and ‘01’ for ‘e’. 


Huffman trees. Let w; be the frequency of the letter c; 
in the given text. It will be convenient to refer to w; as 
the weight of c; or of its external node. To get an effi- 
cient code, we choose short codewords for common let- 
ters. Suppose 6; is the length of the codeword for c;. Then 
the number of bits for encoding the entire text is 


Since 6; is the depth of the leaf c;, P is also known as the 
weighted external path length of the corresponding tree. 


The Huffman tree for the c; minimizes the weighted ex- 
ternal path length. To construct this tree, we start with n 
nodes, one for each letter. At each stage of the algorithm, 
we greedily pick the two nodes with smallest weights and 
make them the children of a new node with weight equal 
to the sum of two weights. We repeat until only one node 
remains. The resulting tree for a collection of nine letters 
with displayed weights is shown in Figure 8. Ties that 


5} [1] [3| 21 
4 


10 


3 
N 


Ss 


Figure 8: The numbers in the external nodes (squares) are the 
weights of the corresponding letters, and the ones in the internal 
nodes (circles) are the weights of these nodes. The Huffman tree 
is full by construction. 


OITTO Je 
TITIO Jw 


Figure 9: The weighted external path length is 15 + 15 + 18 + 
12 +5 + 15 + 24+ 27 + 42 = 173. 


arise during the algorithm are broken arbitrarily. We re- 
draw the tree and order the children of a node as left and 
right child arbitrarily, as shown in Figure 9. 


The algorithm works with a collection N of nodes 
which are the roots of the trees constructed so far. Ini- 
tially, each leaf is a tree by itself. We denote the weight 
of a node by w(u) and use a function EXTRACTMIN that 
returns the node with the smallest weight and, at the same 
time, removes this node from the collection. 
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Tree HUFFMAN 

loop pp = EXTRACTMIN(N); 
if N =Í then return p endif; 
v = EXTRACTMIN(N); 
create node « with children js and v 

and weight w(x) = w(u) + w(v); 

add «x to N 

forever. 


Straightforward implementations use an array or a linked 
list and take time O(n) for each operation involving N. 
There are fewer than 2n extractions of the minimum and 
fewer than n additions, which implies that the total run- 
ning time is O(n”). We will see later that there are better 
ways to implement NV leading to running time O(n log n). 


An inequality. We prepare the proof that the Huffman 
tree indeed minimizes the weighted external path length. 
Let T be a full binary tree with weighted external path 
length P(T). Let A(T) be the set of leaves and let u and 
v be any two leaves with smallest weights. Then we can 
construct a new tree T” with 


(1) set of leaves A(T’) = (A(T) — {u, v}) Ù {K}, 
(2) w(K) = w(u) + wr), 
(3) P(T’) < P(T) — w(u) — w(v), with equality if u 


and v are siblings. 


We now argue that T” really exists. If u and v are siblings 
then we construct T’ from T by removing jy and v and 
declaring their parent, x, as the new leaf. Then 


Figure 10: The increase in the depth of v is compensated by the 
decrease in depth of the leaves in the subtree of ø. 


P(T) = P(T) -—w(u)ô-— w(v) 4 
= P(T) - w(u) - wv), 


w(K)(ô — 1) 


where ô = ô( u) = 6(v) = (K) + 1 is the common depth 
of u and v. Otherwise, assume ô(u) > ô(v) and let o be 


the sibling of jz, which may or may not be a leaf. Exchange 
v and ø. Since the length of the path from the root to ø 
is at least as long as the path to u, the weighted external 
path length can only decrease; see Figure 10. Then do the 
same as in the other case. 


Proof of optimality. The optimality of the Huffman tree 
can now be proved by induction. 


HUFFMAN TREE THEOREM. Let T be the Huffman tree 
and X another tree with the same set of leaves and 
weights. Then P(T) < P(X). 


PROOF. If there are only two leaves then the claim is obvi- 
ous. Otherwise, let u and v be the two leaves selected by 
the algorithm. Construct trees T” and X’ with 


PIT’) = P(T)- w(u) -—w(v), 
P(X’) < P(X)- w(u) - wv). 
T” is the Huffman tree for n — 1 leaves so we can use the 


inductive assumption and get P(T’) < P(X’). It follows 
that 


P(T) 


Í 
x 
+ 
= 
= 
+ 
E 
= 


IA IA 
x 
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Huffman codes are binary codes that correspond to 
Huffman trees as described. They are commonly used to 
compress text and other information. Although Huffman 
codes are optimal in the sense defined above, there are 
other codes that are also sensitive to the frequency of se- 
quences of letters and this way outperform Huffman codes 
for general text. 


Summary. The greedy algorithm for constructing Huff- 
man trees works bottom-up by stepwise merging, rather 
than top-down by stepwise partitioning. If we run the 
greedy algorithm backwards, it becomes very similar to 
dynamic programming, except that it pursues only one of 
many possible partitions. Often this implies that it leads 
to suboptimal solutions. Nevertheless, there are problems 
that exhibit enough structure that the greedy algorithm 
succeeds in finding an optimum, and the scheduling and 
coding problems described above are two such examples. 
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First Homework Assignment 


Write the solution to each problem on a single page. The 
deadline for handing in solutions is September 18. 


Problem 1. (20 points). Consider two sums, X = xı + 
z2 +... + £n and Y = y1 +yo+...+ Ym. Give an 
algorithm that finds indices 2 and j such that swap- 
ping x; with y; makes the two sums equal, that is, 
X — zi + yj = Y — yj + x, if they exist. Analyze 
your algorithm. (You can use sorting as a subroutine. 
The amount of credit depends on the correctness of 
the analysis and the running time of your algorithm.) 


Problem 2. (20 = 10 + 10 points). Consider dis- 
tinct items %1,%2,...,%, With positive weights 
W1,W2,---,Wn such that X; _; w; = 1.0. The 
weighted median is the item xx that satisfies 


So wi <05 and So wi < 0.5. 


Zi LEk LTj>Tk 


(a) Show how to compute the weighted median 
of n items in worst-case time O(n log n) using 
sorting. 

(b) Show how to compute the weighted median in 
worst-case time O(n) using a linear-time me- 
dian algorithm. 


Problem 3. (20 = 6 + 14 points). A game-board has n 
columns, each consisting of a top number, the cost of 
visiting the column, and a bottom number, the maxi- 
mum number of columns you are allowed to jump to 
the right. The top number can be any positive integer, 
while the bottom number is either 1, 2, or 3. The ob- 
jective is to travel from the first column off the board, 
to the right of the nth column. The cost of a game is 
the sum of the costs of the visited columns. 


Assuming the board is represented in a two- 
dimensional array, B[2, n], the following recursive 
procedure computes the cost of the cheapest game: 


int CHEAPEST(int i) 
ifi >nthen return0 endif; 
x = B[|1, i] + CHEAPEST(i + 1); 

y = B[1, i] + CHEAPEST(? + 2); 

z = B[|1, i] + CHEAPEST(i + 3); 

B si 


case B[2,i] = 1: return z; 
B[|2, i] = 2: return min{z, y}; 
B[|2, i] = 3: return min{z, y, z} 
endcase 
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(a) Analyze the asymptotic running time of the pro- 
cedure. 

(b) Describe and analyze a more efficient algorithm 
for finding the cheapest game. 


Problem 4. (20 = 10 + 10 points). Consider a set of n 


intervals [a;, b;] that cover the unit interval, that is, 
[0, 1] is contained in the union of the intervals. 


(a) Describe an algorithm that computes a mini- 
mum subset of the intervals that also covers 
[0, 1]. 

(b) Analyze the running time of your algorithm. 


(For question (b) you get credit for the correctness of 
your analysis but also for the running time of your 
algorithm. In other words, a fast algorithm earns you 
more points than a slow algorithm.) 


Problem 5. (20 = 7 + 7 + 6 points). Let A[1..m] and 


B(1..n] be two strings. 


(a) Modify the dynamic programming algorithm 
for computing the edit distance between A and 
B for the case in which there are only two al- 
lowed operations, insertions and deletions of in- 
dividual letters. 


(b) A (not necessarily contiguous) subsequence of 
A is defined by the increasing sequence of its 
indices, 1 < i4 < ig < ... < ik < m. Use 
dynamic programming to find the longest com- 
mon subsequence of A and B and analyze its 
running time. 


(c 


wm 


What is the relationship between the edit dis- 
tance defined in (a) and the longest common 
subsequence computed in (b)? 


II 


Oo OND 


SEARCHING 


Binary Search Trees 

Red-black Trees 

Amortized Analysis 

Splay Trees 

Second Homework Assignment 


6 Binary Search Trees 


One of the purposes of sorting is to facilitate fast search- 
ing. However, while a sorted sequence stored in a lin- 
ear array is good for searching, it is expensive to add and 
delete items. Binary search trees give you the best of both 
worlds: fast search and fast update. 


Definitions and terminology. We begin with a recursive 
definition of the most common type of tree used in algo- 
rithms. A (rooted) binary tree is either empty or a node 
(the root) with a binary tree as left subtree and binary tree 
as right subtree. We store items in the nodes of the tree. 
It is often convenient to say the items are the nodes. A 
binary tree is sorted if each item is between the smaller or 
equal items in the left subtree and the larger or equal items 
in the right subtree. For example, the tree illustrated in 
Figure 11 is sorted assuming the usual ordering of English 
characters. Terms for relations between family members 
such as child, parent, sibling are also used for nodes in a 
tree. Every node has one parent, except the root which has 
no parent. A leaf or external node is one without children; 
all other nodes are internal. A node v is a descendent of ju 
ifv = wor v is a descendent of a child of u. Symmetri- 
cally, u is an ancestor of v if v is a descendent of u. The 
subtree of u consists of all descendents of u. An edge is a 
parent-child pair. 


ee) 
N 


Figure 11: The parent, sibling and two children of the dark node 
are shaded. The internal nodes are drawn as circles while the 
leaves are drawn as squares. 


The size of the tree is the number of nodes. A binary 
tree is full if every internal node has two children. Every 
full binary tree has one more leaf than internal node. To 
count its edges, we can either count 2 for each internal 
node or 1 for every node other than the root. Either way, 
the total number of edges is one less than the size of the 
tree. A path is a sequence of contiguous edges without 
repetitions. Usually we only consider paths that descend 
or paths that ascend. The length of a path is the number 
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of edges. For every node p, there is a unique path from 
the root to u. The length of that path is the depth of u. 
The height of the tree is the maximum depth of any node. 
The path length is the sum of depths over all nodes, and 
the external path length is the same sum restricted to the 
leaves in the tree. 


Searching. A binary search tree is a sorted binary tree. 
We assume each node is a record storing an item and point- 
ers to two children: 


struct Node{item info; Node * £, xr}; 
typedef Node * Tree. 


Sometimes it is convenient to also store a pointer to the 
parent, but for now we will do without. We can search in 
a binary search tree by tracing a path starting at the root. 


Node * SEARCH(Tree ọ, item x) 
case @ = NULL: return NULL; 
xz < o —> info: return SEARCH(o —> £, 2); 
x = o —> info: return o; 
x > o —> info: return SEARCH(@ —> r, x) 
endcase. 


The running time depends on the length of the path, which 
is at most the height of the tree. Let n be the size. In the 
worst case the tree is a linked list and searching takes time 
O(n). In the best case the tree is perfectly balanced and 
searching takes only time O(log n). 


Insert. To add a new item is similarly straightforward: 
follow a path from the root to a leaf and replace that leaf 
by a new node storing the item. Figure 12 shows the tree 
obtained after adding w to the tree in Figure 11. The run- 


Figure 12: The shaded nodes indicate the path from the root we 
traverse when we insert w into the sorted tree. 


ning time depends again on the length of the path. If the 
insertions come in a random order then the tree is usually 


close to being perfectly balanced. Indeed, the tree is the 
same as the one that arises in the analysis of quicksort. 
The expected number of comparisons for a (successful) 
search is one n-th of the expected running time of quick- 
sort, which is roughly 2 Inn. 


Delete. The main idea for deleting an item is the same 
as for inserting: follow the path from the root to the node 
v that stores the item. 


Case 1. v has no internal node as a child. Remove v. 


Case 2. v has one internal child. Make that child the 
child of the parent of v. 


Case 3. vhastwo internal children. Find the rightmost 
internal node in the left subtree, remove it, and sub- 
stitute it for v, as shown in Figure 13. 


Figure 13: Store J in v and delete the node that used to store J. 


The analysis of the expected search time in a binary search 
tree constructed by a random sequence of insertions and 
deletions is considerably more challenging than if no dele- 
tions are present. Even the definition of a random se- 
quence is ambiguous in this case. 


Optimal binary search trees. Instead of hoping the in- 
cremental construction yields a shallow tree, we can con- 
struct the tree that minimizes the search time. We con- 
sider the common problem in which items have different 
probabilities to be the target of a search. For example, 
some words in the English dictionary are more commonly 
searched than others and are therefore assigned a higher 
probability. Let aj < a2 < ... < an be the items and 
pi the corresponding probabilities. To simplify the discus- 
sion, we only consider successful searches and thus as- 
sume ee pi = 1. The expected number of comparisons 
for a successful search in a binary search tree T' storing 
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the n items is 


orj = Som (+1) 


= 1+) pi: &, 
i=1 


where ô; is the depth of the node that stores a;. C(T) 
is the weighted path length or the cost of T. We study 
the problem of constructing a tree that minimizes the cost. 
To develop an example, let n = 3 and pı = 4, p2 = 
3, p3 = t. Figure 14 shows the five binary trees with 
three nodes and states their costs. It can be shown that the 


aQ ai (a3) a3 (a3) 
a, VT @ oa acm 
a) &@ @) © 


3 


Figure 14: There are five different binary trees of three nodes. 
From left to right their costs are 2 2 $, The first tree and 


the third tree are both optimal. 


number of different binary trees with n nodes is = (2) , 
which is exponential in n. This is far too large to try all 
possibilities, so we need to look for a more efficient way 
to construct an optimum tree. 


Dynamic programming. We write RH for the optimum 
weighted binary search tree of ai, aj41,..-,@;, G for its 
cost, and pl = J ⁄_; pk for the total probability of the 
items in T? . Suppose we know that the optimum tree 
stores item a, in its root. Then the left subtree is ‘ame 
and the right subtree is T 41; The cost of the optimum 
tree is therefore C! = CP"! + Chii + pÍ — pp. Since we 
do not know which item is in the root, we try all possibili- 
ties and find the minimum: 
Ch = amin {CP + Chg, +P; — pr} 

This formula can be translated directly into a dynamic pro- 
gramming algorithm. We use three two-dimensional ar- 
rays, one for the sums of probabilities, pÍ , one for the costs 
of optimum trees, Cc? ‘ and one for the indices of the items 
stored in their roots, R. We assume that the first array has 
already been computed. We initialize the other two arrays 
along the main diagonal and add one dummy diagonal for 
the cost. 


fork=1l1tondo 

Clk,k — 1] =Clk,k] = 0; Rik, k] =k 
endfor; 
Cin + 1,n] = 0. 


We fill the rest of the two arrays one diagonal at a time. 


for l= 2to ndo 
fori=lton—f+1do 
j=i+l-1; Cli,j] =% 
for k =ito jdo 
cost = Cli,k — 1] + Clk +1,3] 
+ pli, j] — plk, k]; 
if cost < Cļ|i, j| then 
Cli, j] = cost, Rfi, j] = k 
endif 
endfor 
endfor 
endfor. 


The main part of the algorithm consists of three nested 
loops each iterating through at most n values. The running 
time is therefore in O(n?). 


Example. 
ities for the data in the earlier example. 


Table 1 shows the partial sums of probabil- 
Table 2 shows 


| 6p |] 1 [2 | 3 | 
Ee ESEAES 
ie ae ae 


Table 1: Six times the partial sums of probabilities used by the 
dynamic programming algorithm. 


the costs and the indices of the roots of the optimum trees 
computed for all contiguous subsequences. The optimum 


Table 2: Six times the costs and the roots of the optimum trees. 


tree can be constructed from œR as follows. The root stores 
the item with index R[1,3] = 1. The left subtree is there- 
fore empty and the right subtree stores a2,a3. The root 
of the optimum right subtree stores the item with index 
R{2, 3] = 2. Again the left subtree is empty and the right 
subtree consists of a single node storing as. 
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Improved running time. Notice that the array R in Ta- 
ble 2 is monotonic, both along TON and along columns. 
Indeed it is possible to prove RII S Rij in every row and 
Ri < Ri +1 in every column. We omit the proof and show 
how the two inequalities can be used to improve the dy- 
namic programming algorithm. Instead of trying all roots 
from 7 through j we restrict the innermost for-loop to 


fork = Rļi, j — 1] to R[i+1,j] do 


The monotonicity property implies that this change does 
not alter the result of the algorithm. The running time of a 
single iteration of the outer for-loop is now 


Uy(n) = Ei (Ri a- RI +1). 


Recall that 7 = i + £ — 1 and note that most terms cancel, 
giving 

e—-1 
=h F 


Uc(n) = Ra-e4+2 (n=£+1) 


< 2M: 


In words, each iteration of the outer for-loop takes only 
time O(n), which implies that the entire algorithm takes 
only time O(n?). 


7 Red-Black Trees 


Binary search trees are an elegant implementation of the 
dictionary data type, which requires support for 


item SEARCH (item), 
void INSERT (item), 
void DELETE (item), 


and possible additional operations. Their main disadvan- 
tage is the worst case time Q(n) for a single operation. 
The reasons are insertions and deletions that tend to get 
the tree unbalanced. It is possible to counteract this ten- 
dency with occasional local restructuring operations and 
to guarantee logarithmic time per operation. 


2-3-4 trees. A special type of balanced tree is the 2-3-4 
tree. Each internal node stores one, two, or three items 
and has two, three, or four children. Each leaf has the 
same depth. As shown in Figure 15, the items in the in- 
ternal nodes separate the items stored in the subtrees and 
thus facilitate fast searching. In the smallest 2-3-4 tree of 
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Figure 15: A 2-3-4 tree of height two. All items are stored in 
internal nodes. 


height h, every internal node has exactly two children, so 
we have 2” leaves and 2” — 1 internal nodes. In the largest 
2-3-4 tree of height h, every internal node has four chil- 
dren, so we have 4” leaves and (4” — 1) /3 internal nodes. 
We can store a 2-3-4 tree in a binary tree by expanding a 
node with ¿ > 1 items and z+ 1 children into 7 nodes each 
with one item, as shown in Figure 16. 


Red-black trees. Suppose we color each edge of a bi- 
nary search tree either red or black. The color is conve- 
niently stored in the lower node of the edge. Such a edge- 
colored tree is a red-black tree if 


(1) there are no two consecutive red edges on any de- 
scending path and every maximal such path ends with 
a black edge; 


(2) all maximal descending paths have the same number 
of black edges. 
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Figure 16: Transforming a 2-3-4 tree into a binary tree. Bold 
edges are called red and the others are called black. 


The number of black edges on a maximal descending path 
is the black height, denoted as bh(@). When we transform 
a 2-3-4 tree into a binary tree as in Figure 16, we get a red- 
black tree. The result of transforming the tree in Figure 15 


4 9 17 


? 


Figure 17: A red-black tree obtained from the 2-3-4 tree in Fig- 
ure 15. 


is shown in Figure 17. 


HEIGHT LEMMA. A red-black tree with n internal nodes 
has height at most 2 log,(n + 1). 


PROOF. The number of leaves is n + 1. Contract each 
red edge to get a 2-3-4 tree with n + 1 leaves. Its height 
is h < log,(n + 1). We have bh(o) = h, and by Rule 
(1) the height of the red-black tree is at most 2bh(0) < 
2 log,(n + 1). 


Rotations. Restructuring a red-black tree can be done 
with only one operation (and its symmetric version): a ro- 
tation that moves a subtree from one side to another, as 
shown in Figure 18. The ordered sequence of nodes in the 
left tree of Figure 18 is 


...,order(A), v, order(B), p, order(C),..., 
and this is also the ordered sequence of nodes in the right 


tree. In other words, a rotation maintains the ordering. 
Function ZIG below implements the right rotation: 


H Zig 
right rotation 


v ma 


AA A 


Figure 18: From left to right a right rotation and from right to 
left a left rotation. 


left rotation 
Zag 


Node *ZIG(Node * 1) 
assert u # NULL andy = u —> £4 NULL; 
pbol=v 


>r V= r= u; return. 

Function ZAG is symmetric and performs a left rotation. 
Occasionally, it is necessary to perform two rotations in 
sequence, and it is convenient to combine them into a sin- 
gle operation referred to as a double rotation, as shown 
in Figure 19. We use a function ZIGZAG to implement a 


double 
right rotation 


ZigZag 


Figure 19: The double right rotation at u is the concatenation of 
a single left rotation at v and a single right rotation at pu. 


double right rotation and the symmetric function ZAGZIG 
to implement a double left rotation. 


Node * ZIGZAG(Node > m) 
u —> €=ZAG(u > l); return ZIG(p1). 


The double right rotation is the composition of two single 
rotations: ZIGZAG(u) = ZIG(j4) o ZAG(v). Remember 
that the composition of functions is written from right to 
left, so the single left rotation of v precedes the single right 
rotation of u. Single rotations preserve the ordering of 
nodes and so do double rotations. 


Insertion. Before studying the details of the restructur- 
ing algorithms for red-black trees, we look at the trees that 
arise in a short insertion sequence, as shown in Figure 20. 
After adding 10, 7, 13, 4, we have two red edges in se- 
quence and repair this by promoting 10 (A). After adding 
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2, we repair the two red edges in sequence by a single ro- 
tation of 7 (B). After adding 5, we promote 4 (C), and after 
adding 6, we do a double rotation of 7 (D). 


Figure 20: Sequence of red-black trees generated by inserting 
the items 10, 7, 13, 4, 2, 5, 6 in this sequence. 


An item xv is added by substituting a new internal node 
for a leaf at the appropriate position. To satisfy Rule (2) 
of the red-black tree definition, color the incoming edge 
of the new node red, as shown in Figure 21. Start the 


Ws 
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Figure 21: The incoming edge of a newly added node is always 
red. 


adjustment of color and structure at the parent v of the new 
node. We state the properties maintained by the insertion 
algorithm as invariants that apply to a node v traced by the 
algorithm. 


INVARIANT I. The only possible violation of the red- 
black tree properties is that of Rule (1) at the node 
v, and if v has a red incoming edge then it has ex- 
actly one red outgoing edge. 


Observe that Invariant I holds right after adding x. We 
continue with the analysis of all the cases that may arise. 
The local adjustment operations depend on the neighbor- 
hood of v. 


Case 1. The incoming edge of v is black. Done. 


Case 2. The incoming edge of v is red. Let ju be the 
parent of v and assume v is left child of p. 


Case 2.1. Both outgoing edges of u are red, as 
in Figure 22. Promote jz. Let v be the parent of 
u and recurse. 


Figure 22: Promotion of u. (The colors of the outgoing edges of 
v may be the other way round). 


Case 2.2. Only one outgoing edge of u is red, 
namely the one from y to v. 


Case 2.2.1. The left outgoing edge of v is 
red, as in Figure 23 to the left. Right rotate 
u. Done. 
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Figure 23: Right rotation of jz to the left and double right rotation 
of u to the right. 
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Case 2.2.2. The right outgoing edge of v 
is red, as in Figure 23 to the right. Double 
right rotate u. Done. 


Case 2 has a symmetric case where left and right are in- 
terchanged. An insertion may cause logarithmically many 
promotions but at most two rotations. 


Deletion. First find the node z that is to be removed. If 
necessary, we substitute the inorder successor for 7 so we 
can assume that both children of 7 are leaves. If 7 is last 
in inorder we substitute symmetrically. Replace m by a 
leaf v, as shown in Figure 24. If the incoming edge of m is 
red then change it to black. Otherwise, remember the in- 
coming edge of v as ‘double-black’, which counts as two 
black edges. Similar to insertions, it helps to understand 
the deletion algorithm in terms of a property it maintains. 


INVARIANT D. The only possible violation of the red- 
black tree properties is a double-black incoming edge 
of v. 
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Figure 24: Deletion of node 7. The dashed edge counts as two 
black edges when we compute the black depth. 


Note that Invariant D holds right after we remove 7. We 
now present the analysis of all the possible cases. The ad- 
justment operation is chosen depending on the local neigh- 
borhood of v. 


Case 1. The incoming edge of v is black. Done. 


Case 2. The incoming edge of v is double-black. Let 
H be the parent and « the sibling of v. Assume v is 
left child of u and note that « is internal. 


Case 2.1. The edge from u to « is black. 


Case 2.1.1. Both outgoing edges of « are 
black, as in Figure 25. Demote u. Recurse 
forv = u. 


Figure 25: Demotion of ju. 


Case 2.1.2. The right outgoing edge of «k 
is red, as in Figure 26 to the left. Change 
the color of that edge to black and left ro- 
tate u. Done. 


Figure 26: Left rotation of u to the left and double left rotation 
of u to the right. 


Case 2.1.3. The right outgoing edge of 
k is black, as in Figure 26 to the right. 
Change the color of the left outgoing edge 
to black and double left rotate u. Done. 


Case 2.2. The edge from ņ to « is red, as in Fig- 
ure 27. Left rotate u. Recurse for v. 


Figure 27: Left rotation of u. 


Case 2 has a symmetric case in which v is the right child of 
u. Case 2.2 seems problematic because it recurses without 
moving v any closer to the root. However, the configura- 
tion excludes the possibility of Case 2.2 occurring again. 
If we enter Cases 2.1.2 or 2.1.3 then the termination is im- 
mediate. If we enter Case 2.1.1 then the termination fol- 
lows because the incoming edge of u is red. The deletion 
may cause logarithmically many demotions but at most 
three rotations. 


Summary. The red-black tree is an implementation 
of the dictionary data type and supports the operations 
search, insert, delete in logarithmic time each. An inser- 
tion or deletion requires the equivalent of at most three 
single rotations. The red-black tree also supports finding 
the minimum, maximum and the inorder successor, prede- 
cessor of a given node in logarithmic time each. 
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8 Amortized Analysis 


Amortization is an analysis technique that can influence 
the design of algorithms in a profound way. Later in this 
course, we will encounter data structures that owe their 
very existence to the insight gained in performance due to 
amortized analysis. 


Binary counting. We illustrate the idea of amortization 
by analyzing the cost of counting in binary. Think of an 
integer as a linear array of bits, n = J- ;>ọ Ali] - 2’. The 
following loop keeps incrementing the integer stored in A. 


loop i = 0; 
while Ali] = 1 do Afi] = 0; i++ endwhile; 
Afi] = 1. 


forever. 


We define the cost of counting as the total number of bit 
changes that are needed to increment the number one by 
one. What is the cost to count from 0 to n? Figure 28 
shows that counting from 0 to 15 requires 26 bit changes. 
Since n takes only 1 + |log, n] bits or positions in A, 


5 0000000000000000 
4 0000000000000 0 0j0 
3 OD0DDIDIDODO1LI11Iii isl 
2 000/01 1 1]1}0 0 0j0/1 1 1/1 
1 0/0} 1/1/00 1 | 1) 0/0) 1/1/0/0| 1/1 
o Ooo oo ooo 


Figure 28: The numbers are written vertically from top to bot- 
tom. The boxed bits change when the number is incremented. 


a single increment does at most 2 + log, n steps. This 
implies that the cost of counting from 0 to n is at most 
n logs n + 2n. Even though the upper bound of 2 + log, n 
is almost tight for the worst single step, we can show that 
the total cost is much less than n times that. We do this 
with two slightly different amortization methods referred 
to as aggregation and accounting. 


Aggregation. The aggregation method takes a global 
view of the problem. The pattern in Figure 28 suggests 
we define b; equal to the number of Is and t; equal to 
the number of trailing 1s in the binary notation of 7. Ev- 
ery other number has no trailing 1, every other number 
of the remaining ones has one trailing 1, etc. Assuming 
n = 2} — 1, we therefore have exactly j — 1 trailing 1s 
for 2*-J = (n + 1)/2/ integers between 0 and n — 1. The 


26 


total number of bit changes is therefore 


n-1 k i 
T(n) = Y(t +1) = CERDDI- 
i=0 j=1 


We use index transformation to show that the sum on the 
right is less than 2: 
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Hence the cost is T(n) < 2(n + 1). The amortized cost 
Ta) Which is about 2. 


n 


per operation is 


Accounting. The idea of the accounting method is to 
charge each operation what we think its amortized cost is. 
If the amortized cost exceeds the actual cost, then the sur- 
plus remains as a credit associated with the data structure. 
If the amortized cost is less than the actual cost, the accu- 
mulated credit is used to pay for the cost overflow. Define 
the amortized cost of a bit change 0 — 1 as $2 and that 
of 1 — 0 as $0. When we change 0 to 1 we pay $1 for 
the actual expense and $1 stays with the bit, which is now 
1. This $1 pays for the (later) cost of changing the 1 to 0. 
Each increment has amortized cost $2, and together with 
the money in the system, this is enough to pay for all the 
bit changes. The cost is therefore at most 2n. 


We see how a little trick, like making the 0 — 1 changes 
pay for the 1 — 0 changes, leads to a very simple analysis 
that is even more accurate than the one obtained by aggre- 
gation. 


Potential functions. We can further formalize the amor- 
tized analysis by using a potential function. The idea is 
similar to accounting, except there is no explicit credit 
saved anywhere. The accumulated credit is an expres- 
sion of the well-being or potential of the data structure. 
Let c; be the actual cost of the 7-th operation and D; the 
data structure after the i-th operation. Let ®; = ®(D;) 
be the potential of D;, which is some numerical value 
depending on the concrete application. Then we define 
ai = ci + ®; — ®;_1 as the amortized cost of the i-th 


operation. The sum of amortized costs of n operations is 


n 
y ay = 
i=1 


(ci + ®; — i1) 


® 
Ci + On — o. 


N 
i=1 
n 
i=1 
We aim at choosing the potential such that ®g = 0 and 
n > 0 because then we get >a; > D> ci. In words, 
the sum of amortized costs covers the sum of actual costs. 
To apply the method to binary counting we define the po- 


tential equal to the number of 1s in the binary notation, 
®; = b;. It follows that 


®-—@-1 = bi- bi- 
= (bj-1 —%&~-1 + 1) — bi—1 


The actual cost of the i-th operation is c; = 1 + ti—1, 
and the amortized cost is aj = c; + ®; — ®;_1 = 2. 
We have Po = 0 and ®,, > 0 as desired, and therefore 
X ci < SS a; = 2n, which is consistent with the analysis 
of binary counting with the aggregation and the account- 
ing methods. 


2-3-4 trees. As amore complicated application of amor- 
tization we consider 2-3-4 trees and the cost of restructur- 
ing them under insertions and deletions. We have seen 
2-3-4 trees earlier when we talked about red-black trees. 
A set of keys is stored in sorted order in the internal nodes 
of a 2-3-4 tree, which is characterized by the following 
rules: 


(1) each internal node has 2 < d < 4 children and stores 
d — 1 keys; 


(2) all leaves have the same depth. 


As for binary trees, being sorted means that the left-to- 
right order of the keys is sorted. The only meaningful def- 
inition of this ordering is the ordered sequence of the first 
subtree followed by the first key stored in the root followed 
by the ordered sequence of the second subtree followed by 
the second key, etc. 


To insert a new key, we attach a new leaf and add the key 
to the parent v of that leaf. All is fine unless v overflows 
because it now has five children. If it does, we repair the 
violation of Rule (1) by climbing the tree one node at a 
time. We call an internal node non-saturated if it has fewer 
than four children. 
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Case 1. vhas five children and a non-saturated sibling 
to its left or right. Move one child from v to that 
sibling, as in Figure 29. 


$6 $1 $0 


Figure 29: The overflowing node gives one child to a non- 
saturated sibling. 


Case 2. v has five children and no non-saturated sib- 
ling. Split v into two nodes and recurse for the parent 
of v, as in Figure 30. If v has no parent then create a 
new root whose only children are the two nodes ob- 
tained from v. 


$3 $6 


CA 


$0 $1 
Figure 30: The overflowing node is split into two and the parent 
is treated recursively. 
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Deleting a key is done is a similar fashion, although there 
we have to battle with nodes v that have too few children 
rather than too many. Let v have only one child. We repair 
Rule (1) by adopting a child from a sibling or by merging 
v with a sibling. In the latter case the parent of v looses a 
child and needs to be visited recursively. The two opera- 
tions are illustrated in Figures 31 and 32. 


$3 


Figure 31: The underflowing node receives one child from a sib- 
ling. 


Amortized analysis. The worst case for inserting a new 
key occurs when all internal nodes are saturated. The in- 
sertion then triggers logarithmically many splits. Sym- 
metrically, the worst case for a deletion occurs when all 


$0 $1 
$1 / $4 —> $0 


Q 7 


Figure 32: The underflowing node is merged with a sibling and 
the parent is treated recursively. 


internal nodes have only two children. The deletion then 
triggers logarithmically many mergers. Nevertheless, we 
can show that in the amortized sense there are at most a 
constant number of split and merge operations per inser- 
tion and deletion. 


We use the accounting method and store money in the 
internal nodes. The best internal nodes have three children 
because then they are flexible in both directions. They 
require no money, but all other nodes are given a posi- 
tive amount to pay for future expenses caused by split and 
merge operations. Specifically, we store $4, $1, $0, $3, 
$6 in each internal node with 1, 2, 3, 4, 5 children. As il- 
lustrated in Figures 29 and 31, an adoption moves money 
only from v to its sibling. The operation keeps the total 
amount the same or decreases it, which is even better. As 
shown in Figure 30, a split frees up $5 from v and spends 
at most $3 on the parent. The extra $2 pay for the split 
operation. Similarly, a merger frees $5 from the two af- 
fected nodes and spends at most $3 on the parent. This 
is illustrated in Figure 32. An insertion makes an initial 
investment of at most $3 to pay for creating a new leaf. 
Similarly, a deletion makes an initial investment of at most 
$3 for destroying a leaf. If we charge $2 for each split and 
each merge operation, the money in the system suffices to 
cover the expenses. This implies that for n insertions and 
deletions we get a total of at most 3a split and merge oper- 
ations. In other words, the amortized number of split and 
merge operations is at most 3. 


Recall that there is a one-to-one correspondence be- 
tween 2-3-4 tree and red-black trees. We can thus trans- 
late the above update procedure and get an algorithm for 
red-black trees with an amortized constant restructuring 
cost per insertion and deletion. We already proved that for 
red-black trees the number of rotations per insertion and 
deletion is at most a constant. The above argument im- 
plies that also the number of promotions and demotions is 
at most a constant, although in the amortized and not in 
the worst-case sense as for the rotations. 
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9 Splay Trees 


Splay trees are similar to red-black trees except that they 
guarantee good shape (small height) only on the average. 
They are simpler to code than red-black trees and have the 
additional advantage of giving faster access to items that 
are more frequently searched. The reason for both is that 
splay trees are self-adjusting. 


Self-adjusting binary search trees. Instead of explic- 
itly maintaining the balance using additional information 
(such as the color of edges in the red-black tree), splay 
trees maintain balance implicitly through a self-adjusting 
mechanism. Good shape is a side-effect of the operations 
that are applied. These operations are applied while splay- 
ing a node, which means moving it up to the root of the 
tree, as illustrated in Figure 33. A detailed analysis will 


Figure 33: The node storing 1 is splayed using three single rota- 
tions. 


reveal that single rotations do not imply good amortized 
performance but combinations of single rotations in pairs 
do. Aside from double rotations, we use roller-coaster 
rotations that compose two single left or two single right 
rotations, as shown in Figure 35. The sequence of the two 
single rotations is important, namely first the higher then 
the lower node. Recall that ZIG(«) performs a single right 
rotation and returns the new root of the rotated subtree. 
The roller-coaster rotation to the right is then 


Node «* ZIGZIG(Node *k) 
return ZIG(ZIG(k)). 


Function ZAGZAG is symmetric, exchanging left and 
right, and functions ZIGZAG and ZAGZIG are the two 
double rotations already used for red-black trees. 


Splay. A splay operation finds an item and uses rotations 
to move the corresponding node up to the root position. 
Whenever possible, a double rotation or a roller-coaster 
rotation is used. We dispense with special cases and show 
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Function SPLAY for the case the search item z is less than 
the item in the root. 


if x < ọ— infothen u= ọ—> £, 
if x < u — info then 
u —> £ = SPLAY (u > £, x); 
return ZIGZIG(o) 
elseif x > u — info then 
u > r = SPLAY (u > r, z); 
return ZIGZAG(Q) 
else 
return ZIG(o) 
endif. 


If x is stored in one of the children of ọ then it is moved 
to the root by a single rotation. Otherwise, it is splayed 
recursively to the third level and moved to the root either 
by a double or a roller-coaster rotation. The number of 
rotation depends on the length of the path from ọ to zx. 
Specifically, if the path is 2 edges long then z is splayed in 
|i/2| double and roller-coaster rotations and zero or one 
single rotation. In the worst case, a single splay operation 
takes almost as many rotations as there are nodes in the 
tree. We will see shortly that the amortized number of 
rotations is at most logarithmic in the number of nodes. 


Amortized cost. Recall that the amortized cost of an op- 
eration is the actual cost minus the cost for work put into 
improving the data structure. To analyze the cost, we use a 
potential function that measures the well-being of the data 
structure. We need definitions: 


the size s(v) is the number of descendents of node v, in- 
cluding v, 


the balance 3(v) is twice the floor of the binary logarithm 
of the size, G(v) = 2|log, s(v) |, 


the potential ® of a tree or a collection of trees is the sum 
of balances over all nodes, ® = X` G(v), 


the actual cost c; of the i-th splay operation is 1 plus the 
number of single rotations (counting a double or 
roller-coaster rotation as two single rotations). 


the amortized cost a; of the i-th splay operation is a; = 
Cy + P; = ®;_ 4. 


We have Po = 0 for the empty tree and ®; > 0 in general. 
This implies that the total actual cost does not exceed the 
total amortized cost, X` c; = X a; — ®, + o < So aj. 


To get a feeling for the potential, we compute ® for 
the two extreme cases. Note first that the integral of the 


natural logarithm is [Ina = «lnz — x and therefore 
flogs = xlogyx — x/ln2. In the extreme unbal- 
anced case, the balance of the i-th node from the bottom 
is 2| log, i| and the potential is 


22l [loggi] = 2nlog,n— O(n). 


In the balanced case, we bound ® from above by 2U(n), 
where U (n) = 2U(%)+log, n. We prove that U (n) < 2n 
for the case when n = 2*. Consider the perfectly balanced 
tree with n leaves. The height of the tree is k = logy n. 
We encode the term log, n of the recurrence relation by 
drawing the hook-like path from the root to the right child 
and then following left edges until we reach the leaf level. 
Each internal node encodes one of the recursively surfac- 
ing log-terms by a hook-like path starting at that node. The 
paths are pairwise edge-disjoint, which implies that their 
total length is at most the number of edges in the tree, 
which is 2n — 2. 


Investment. The main part of the amortized time analy- 
sis is a detailed study of the three types of rotations: sin- 
gle, roller-coaster, and double. We write 3(v) for the bal- 
ance of a node v before the rotation and /3’(v) for the bal- 
ance after the rotation. Let v be the lowest node involved 
in the rotation. The goal is to prove that the amortized 
cost of a roller-coaster and a double rotation is at most 
3[6’(v) — B(v)] each, and that of a single rotation is at 
most 1 + 3[8' (v) — 3(v)]. Summing these terms over the 
rotations of a splay operation gives a telescoping series in 
which all terms cancel except the first and the last. To this 
we add | for the at most one single rotation and another 1 
for the constant cost in definition of actual cost. 


INVESTMENT LEMMA. The amortized cost of splaying a 
node v in a tree o is at most 2 + 3[G(e) — A(v)}. 


Before looking at the details of the three types of rota- 
tions, we prove that if two siblings have the same balance 
then their common parent has a larger balance. Because 
balances are even integers this means that the balance of 
the parent exceeds the balance of its children by at least 2. 


BALANCE LEMMA. If u has children v,« and G(v) = 


p(k) = 2 then B(u) > p +2. 


PROOF. By definition (v) = 
s(v) > 28/2. We have s(u) = 
and therefore G(j1) > 8+2. 


So s(v)| and therefore 
+s(v) + s(x) > 287, 
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Single rotation. The amortized cost of a single rotation 
shown in Figure 34 is 1 for performing the rotation plus 
the change in the potential: 


1+ 6'(v)4 
< 143] @)= 
because (3'() < (u) and (v) < 8'(V). 


MAT AB 


Figure 34: The size of u decreases and that of v increases from 
before to after the rotation. 


a 
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Roller-coaster rotation. The amortized cost of a roller- 
coaster rotation shown in Figure 35 is 


a = 2+ß'(v)+8'(u)+8 (s) 
— B(v) — b(n) — blr) 
< 24 2[8’(v) — bpw) 


because 8' (x) < (x), B'(u) < P'O), and pi) < B(u). 
We distinguish two cases to prove that a is bounded from 
above by 3[8'(v) — (v)]. In both cases, the drop in the 


Figure 35: If in the middle tree the balance of v is the same as 
the balance of u then by the Balance Lemma the balance of « is 
less than that common balance. 


potential pays for the two single rotations. 


Case p'(v) > B(v). The difference between the balance 
of v before and after the roller-coaster rotation is at 
least 2. Hence a < 3[0’(v) — B(v)]. 


Case '(v) = (v) = p. Then the balances of nodes v 
and js in the middle tree in Figure 35 are also equal 
to 3. The Balance Lemma thus implies that the bal- 
ance of «x in that middle tree is at most 8 — 2. But 
since the balance of « after the roller-coaster rotation 
is the same as in the middle tree, we have 3’(K) < 2. 
Hence a < 0 = 3[8' (v) — A(v))]. 


Double rotation. The amortized cost of a double rota- 
tion shown in Figure 36 is 


2+ B'(v) +p (u) + BK) 
— By) — plu) — B(s) 
< 2+ [6"(v) - 6) 


because 8'(k) < (kK) and 8'(u) < G(s). We again dis- 
tinguish two cases to prove that a is bounded from above 
by 3[8' (v) — B(v)]. In both cases, the drop in the potential 
pays for the two single rotations. 


a = 


Case '(v) > (v). The difference is at least 2, which 
implies a < 3[8'(v) — G(v)], as before. 


case f'(v) = Av) = B. Then B(u) = A(x) = B. We 
have 8'(u) < B'(v) or B'(K) < 8’ (v) by the Balance 
Lemma. Hence a < 0 = 3[8’(v) — G(v)}. 


Woy 


Figure 36: In a double rotation, the sizes of u and « decrease 
from before to after the operation. 


Dictionary operations. In summary, we showed that the 
amortized cost of splaying a node v in a binary search tree 
with root o is at most 1+ 3[3(@) — B(v)]. We now use this 
result to show that splay trees have good amortized perfor- 
mance for all standard dictionary operations and more. 


To access an item we first splay it to the root and return 
the root even if it does not contain x. The amortized cost 


is O(5(g)). 


Given an item x, we can split a splay tree into two, 
one containing all items smaller than or equal to x and the 
other all items larger than xv, as illustrated in Figure 37. 
The amortized cost is the amortized cost for splaying plus 


(x (x 


A 


Figure 37: After splaying {x to the root, we split the tree by un- 
linking the right subtree. 
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the increase in the potential, which we denote as ®’ — ©. 
Recall that the potential of a collection of trees is the sum 
of the balances of all nodes. Splitting the tree decreases 
the number of descendents and therefore the balance of 
the root, which implies that 6’ — @ < 0. It follows that 
the amortized cost of a split operation is less than that of a 
splay operation and therefore in O(8(ọ)). 


Two splay trees can be joined into one if all items in 
one tree are smaller than all items in the other tree, as il- 
lustrated in Figure 38. The cost for splaying the maximum 


Figure 38: We first splay the maximum in the tree with the 
smaller items and then link the two trees. 


in the first tree is O(G(01)). The potential increase caused 
by linking the two trees is 


-P < 2|logz(s(21) + s(22))] 
< 2log, s(01) + 2 log, s(02). 


The amortized cost of joining is thus O(8(01) + G(@2)). 


To insert a new item, x, we split the tree. If x is al- 
ready in the tree, we undo the split operation by linking 
the two trees. Otherwise, we make the two trees the left 
and right subtrees of a new node storing x. The amortized 
cost for splaying is O(3(g)). The potential increase caused 
by linking is 


6’ ® 


IA 


2|logs(s(o1) + s(@2) + 1)] 
= b(o). 


The amortized cost of an insertion is thus O(/3(@)). 


To delete an item, we splay it to the root, remove the 
root, and join the two subtrees. Removing x decreases the 
potential, and the amortized cost of joining the two sub- 
trees is at most O(G(g)). This implies that the amortized 
cost of a deletion is at most O(3(@)). 


Weighted search. A nice property of splay trees not 
shared by most other balanced trees is that they automat- 
ically adapt to biased search probabilities. It is plausible 
that this would be the case because items that are often 
accessed tend to live at or near the root of the tree. The 
analysis is somewhat involved and we only state the re- 
sult. Each item or node has a positive weight, w(v) > 0, 


and we define W = $, w(v). We have the following 
generalization of the Investment Lemma, which we state 
without proof. 


WEIGHTED INVESTMENT LEMMA. The amortized cost 
of splaying a node v in a tree with total weight W 
is at most 2 + 3 log,(W/w(v)). 


It can be shown that this result is asymptotically best pos- 
sible. In other words, the amortized search time in a splay 
tree is at most a constant times the optimum, which is 
what we achieve with an optimum weighted binary search 
tree. In contrast to splay trees, optimum trees are expen- 
sive to construct and they require explicit knowledge of 
the weights. 
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Second Homework Assignment 


Write the solution to each problem on a single page. The 
deadline for handing in solutions is October 02. 


Problem 1. (20 = 12 + 8 points). Consider an array 
A{1..n] for which we know that A[1] > A[2] and 
Aln— 1] < Afn]. We say that i is a local minimum if 
Ali — 1] > Alt] < Aļi + 1]. Note that A has at least 
one local minimum. 


(a) We can obviously find a local minimum in time 
O(n). Describe a more efficient algorithm that 
does the same. 


(b) Analyze your algorithm. 


Problem 2. (20 points). A vertex cover for a tree is a sub- 
set V of its vertices such that each edge has at least 
one endpoint in V. It is minimum if there is no other 
vertex cover with a smaller number of vertices. Given 
a tree with n vertices, describe an O(n)-time algo- 
rithm for finding a minimum vertex cover. (Hint: use 
dynamic programming or the greedy method.) 


Problem 3. (20 points). Consider a red-black tree formed 
by the sequential insertion of n > 1 items. Argue that 
the resulting tree has at least one red edge. 


[Notice that we are talking about a red-black tree 
formed by insertions. Without this assumption, the 
tree could of course consist of black edges only.] 


Problem 4. (20 points). Prove that 2n rotations suffice to 
transform any binary search tree into any other binary 
search tree storing the same n items. 


Problem 5. (20 = 5+ 5 + 5 + 5 points). Consider a 
collection of items, each consisting of a key and a 
cost. The keys come from a totally ordered universe 
and the costs are real numbers. Show how to maintain 
a collection of items under the following operations: 


(a) ADD(k,c): assuming no item in the collection 
has key k yet, add an item with key k and cost 
c to the collection; 

(b) REMOVE(K): remove the item with key k from 
the collection; 

(c) MAX(k1,k2): assuming kı < ko, report the 
maximum cost among all items with keys k € 
[ki1, k2]. 


(d) COUNT(ci, c2): assuming cı < c2, report the 
number of items with cost c € [c1, c2]; 


Each operation should take at most O(log n) time in 
the worst case, where n is the number of items in the 
collection when the operation is performed. 
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10 Heaps and Heapsort 


A heap is a data structure that stores a set and allows fast 
access to the item with highest priority. It is the basis of 
a fast implementation of selection sort. On the average, 
this algorithm is a little slower than quicksort but it is not 
sensitive to the input ordering or to random bits and runs 
about as fast in the worst case as on the average. 


Priority queues. A data structure implements the prior- 
ity queue abstract data type if it supports at least the fol- 
lowing operations: 


void INSERT (item), 
item FINDMIN (void), 
void DELETEMIN (void). 


The operations are applied to a set of items with priori- 
ties. The priorities are totally ordered so any two can be 
compared. To avoid any confusion, we will usually refer 
to the priorities as ranks. We will always use integers as 
priorities and follow the convention that smaller ranks rep- 
resent higher priorities. In many applications, FINDMIN 
and DELETEMIN are combined: 


void EXTRACTMIN(void) 
r = FINDMIN; DELETEMIN; returnr. 


Function EXTRACTMIN removes and returns the item 
with smallest rank. 


Heap. A heap is a particularly compact priority queue. 
We can think of it as a binary tree with items stored in the 
internal nodes, as in Figure 39. Each level is full, except 


Ar? 


Figure 39: Ranks increase or, more precisely, do not decrease 
from top to bottom. 


© 


possibly the last, which is filled from left to right until 
we run out of items. The items are stored in heap-order: 
every node u has a rank larger than or equal to the rank of 
its parent. Symmetrically, 4 has a rank less than or equal 
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to the ranks of both its children. As a consequence, the 
root contains the item with smallest rank. 


We store the nodes of the tree in a linear array, level 
by level from top to bottom and each level from left to 
right, as shown in Figure 40. The embedding saves ex- 


Figure 40: The binary tree is layed out in a linear array. The root 
is placed in A[1], its children follow in A[2] and A[3], etc. 


plicit pointers otherwise needed to establish parent-child 
relations. Specifically, we can find the children and par- 
ent of a node by index computation: the left child of A[i] 
is A[2i], the right child is A[2i + 1], and the parent is 
A[|i/2|]. The item with minimum rank is stored in the 
first element: 


item FINDMIN(int n) 
assert n> 1; return A[l]. 


Since the index along a path at least doubles each step, 
paths can have length at most log, n. 


Deleting the minimum. We first study the problem of 
repairing the heap-order if it is violated at the root, as 
shown in Figure 41. Let n be the length of the array. We 


Figure 41: The root is exchanged with the smaller of its two 
children. The operation is repeated along a single path until the 
heap-order is repaired. 


repair the heap-order by a sequence of swaps along a sin- 
gle path. Each swap is between an item and the smaller of 
its children: 


void SIFT-DN(int i, n) 
if 2i < n then 
k = arg min{ A[2i], A[2i + 1]} 
if A[k] < Afi] then Swar(i, k); 
SIFT-DN(k, n) 
endif 
endif. 


Here we assume that A[n + 1] is defined and larger than 
Al[n]. Since a path has at most log, n edges, the time to re- 
pair the heap-order takes time at most O(log n). To delete 
the minimum we overwrite the root with the last element, 
shorten the heap, and repair the heap-order: 


void DELETEMIN(int *n) 
Al] = Af*n]; *n——; SIFT-DN(1, *n). 


Instead of the variable that stores n, we pass a pointer to 
that variable, xn, in order to use it as input and output 
parameter. 


Inserting. Consider repairing the heap-order if it is vio- 
lated at the last position of the heap. In this case, the item 
moves up the heap until it reaches a position where its rank 
is at least as large as that of its parent. 


void SIFT-UP(int i) 
ifi>2thenk= |i/2]; 
if Ali] < A[k] then Swap(i, k); 
SIFT-UP(k) 
endif 
endif. 


An item is added by first expanding the heap by one ele- 
ment, placing the new item in the position that just opened 
up, and repairing the heap-order. 


void INSERT(int xn, item zx) 
xn++; Alxn] = x; SIFT-UP(*n). 


A heap supports FINDMIN in constant time and INSERT 
and DELETEMIN in time O(log n) each. 


Sorting. Priority queues can be used for sorting. The 
first step throws all items into the priority queue, and the 
second step takes them out in order. Assuming the items 
are already stored in the array, the first step can be done 
by repeated heap repair: 


fori = 1 to n do SIFT-UP(i) endfor. 
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In the worst case, the ¿-th item moves up all the way to 
the root. The number of exchanges is therefore at most 
X; loggi < nlogọn. The upper bound is asymptot- 
ically tight because half the terms in the sum are at least 
logy 5 = loga n— 1. It is also possible to construct the ini- 
tial heap in time O(n) by building it from bottom to top. 
We modify the first step accordingly, and we implement 
the second step to rearrange the items in sorted order: 


void HEAPSORT(int n) 
for i = n downto 1 do SIFT-DN(i, n) endfor; 
for i = n downto 1 do 
SwaP(i, 1); SIFT-DN(1,7— 1) 
endfor. 


At each step of the first for-loop, we consider the sub- 
tree with root Ali]. At this moment, the items in the left 
and right subtrees rooted at A[22] and A[2¢ + 1] are al- 
ready heaps. We can therefore use one call to function 
SIFT-DN to make the subtree with root A[i] a heap. We 
will prove shortly that this bottom-up construction of the 
heap takes time only O(n). Figure 42 shows the array 
after each iteration of the second for-loop. Note how 
the heap gets smaller by one element each step. A sin- 
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Figure 42: Each step moves the last heap element to the root and 
thus shrinks the heap. The circles mark the items involved in the 
sift-down operation. 


gle sift-down operation takes time O(log n), and in total 
HEAPSORT takes time O(n logn). In addition to the in- 
put array, HEAPSORT uses a constant number of variables 


and memory for the recursion stack used by SIFT-DN. 
We can save the memory for the stack by writing func- 
tion SIFT-DN as an iteration. The sort can be changed to 
non-decreasing order by reversing the order of items in the 
heap. 


Analysis of heap construction. We return to proving 
that the bottom-up approach to constructing a heap takes 
only O(n) time. Assuming the worst case, in which ev- 
ery node sifts down all the way to the last level, we draw 
the swaps as edges in a tree; see Figure 43. To avoid 
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Figure 43: Each node generates a path that shares no edges with 
the paths of the other nodes. 
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drawing any edge twice, we always first swap to the right 
and then continue swapping to the left until we arrive at 
the last level. This introduces only a small inaccuracy in 
our estimate. The paths cover each edge once, except for 
the edges on the leftmost path, which are not covered at 
all. The number of edges in the tree is n — 1, which im- 
plies that the total number of swaps is less than n. Equiv- 
alently, the amortized number of swaps per item is less 
than 1. There is a striking difference in time-complexity 
to sorting, which takes an amortized number of about 
logy n comparisons per item. The difference between 1 
and log, n may be interpreted as a measure of how far 
from sorted a heap-ordered array still is. 
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11 Fibonacci Heaps 


The Fibonacci heap is a data structure implementing the 
priority queue abstract data type, just like the ordinary 
heap but more complicated and asymptotically faster for 
some operations. We first introduce binomial trees, which 
are special heap-ordered trees, and then explain Fibonacci 
heaps as collections of heap-ordered trees. 


Binomial trees. The binomial tree of height h is a tree 
obtained from two binomial trees of height h — 1, by link- 
ing the root of one to the other. The binomial tree of height 
0 consists of a single node. Binomial trees of heights up 
to 4 are shown in Figure 44. Each step in the construc- 


JN J 2 


Figure 44: Binomial trees of heights 0, 1, 2, 3, 4. Each tree is 
obtained by linking two copies of the previous tree. 


tion increases the height by one, increases the degree (the 
number of children) of the root by one, and doubles the 
size of the tree. It follows that a binomial tree of height h 
has root degree h and size 2”. The root has the largest de- 
gree of any node in the binomial tree, which implies that 
every node in a binomial tree with n nodes has degree at 
most log, n. 


To store any set of items with priorities, we use a small 
collection of binomial trees. For an integer n, let n; be 
the z-th bit in the binary notation, so we can write n = 
Ys ni2’. To store n items, we use a binomial tree of 
size 2‘ foreach n; = 1. The total number of binomial trees 
is thus the number of 1’s in the binary notation of n, which 
is at most log,(n + 1). The collection is referred to as a 
binomial heap. The items in each binomial tree are stored 
in heap-order. There is no specific relationship between 
the items stored in different binomial trees. The item with 
minimum key is thus stored in one of the logarithmically 
many roots, but it is not prescribed ahead of time in which 
one. An example is shown in Figure 45 where 1li9 = 
1011, items are stored in three binomial trees with sizes 
8, 2, and 1. In order to add a new item to the set, we create 
a new binomial tree of size 1 and we successively link 
binomial trees as dictated by the rules of adding | to the 
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Figure 45: Adding the shaded node to a binomial heap consisting 
of three binomial trees. 


binary notation of n. In the example, we get 10112 +12 = 
11002. The new collection thus consists of two binomial 
trees with sizes 8 and 4. The size 8 tree is the old one, and 
the size 4 tree is obtained by first linking the two size 1 
trees and then linking the resulting size 2 tree to the old 
size 2 tree. All this is illustrated in Figure 45. 


Fibonacci heaps. A Fibonacci heap is a collection of 
heap-ordered trees. Ideally, we would like it to be a col- 
lection of binomial trees, but we need more flexibility. It 
will be important to understand how exactly the nodes of a 
Fibonacci heap are connected by pointers. Siblings are or- 
ganized in doubly-linked cyclic lists, and each node has a 
pointer to its parent and a pointer to one of its children, as 
shown in Figure 46. Besides the pointers, each node stores 


Figure 46: The Fibonacci heap representation of the first collec- 
tion of heap-ordered trees in Figure 45. 


a key, its degree, and a bit that can be used to mark or un- 
mark the node. The roots of the heap-ordered trees are 
doubly-linked in a cycle, and there is an explicit pointer to 
the root that stores the item with the minimum key. Figure 
47 illustrates a few basic operations we perform on a Fi- 
bonacci heap. Given two heap-ordered trees, we link them 
by making the root with the bigger key the child of the 
other root. To unlink a heap-ordered tree or subtree, we 
remove its root from the doubly-linked cycle. Finally, to 
merge two cycles, we cut both open and connect them at 
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Figure 47: Cartoons for linking two trees, unlinking a tree, and 
merging two cycles. 


their ends. Any one of these three operations takes only 
constant time. 


Potential function. A Fibonacci heap supports a vari- 
ety of operations, including the standard ones for priority 
queues. We use a potential function to analyze their amor- 
tized cost applied to an initially empty Fibonacci heap. 
Letting r; be the number of roots in the root cycle and 
m; the number of marked nodes, the potential after the 
i-th operation is ®; = r; +2m,;. When we deal with a col- 
lection of Fibonacci heaps, we define its potential as the 
sum of individual potentials. The initial Fibonacci heap is 
empty, so Po = 0. As usual, we let c; be the actual cost 
and a; = ci + ®; — ®;_1 the amortized cost of the i-th 
operation. Since ®ọ = 0 and ®; > 0 for all 7, the actual 
cost is less than the amortized cost: 


For some of the operations, it is fairly easy to compute the 
amortized cost. We get the minimum by returning the key 
in the marked root. This operation does not change the po- 
tential and its amortized and actual cost is a; = c; = 1. 
We meld two Fibonacci heaps, Hı and Hg, by first merg- 
ing the two root circles and second adjusting the pointer to 
the minimum key. We have 


ri—1(A1) + ri-1 (A), 
mi—1(H1) + mi-1(H2), 


which implies that there is no change in potential. The 
amortized and actual cost is therefore a; = c; = 1. We 
insert a key into a Fibonacci heap by first creating a new 
Fibonacci heap that stores only the new key and second 
melding the two heaps. We have one more node in the 
root cycle so the change in potential is ®; — ®;_, = 1. 
The amortized cost is therefore a; = ci + 1 = 2. 


39 


Deletemin. Next we consider the somewhat more in- 
volved operation of deleting the minimum key, which is 
done in four steps: 


Step 1. Remove the node with minimum key from the 
root cycle. 


Step 2. Merge the root cycle with the cycle of children 
of the removed node. 


Step 3. As long as there are two roots with the same 
degree link them. 


Step 4. Recompute the pointer to the minimum key. 


For Step 3, we use a pointer array R. Initially, R[i] = 
NULL for each 7. For each root @ in the root cycle, we 
execute the following iteration. 


i =o —> degree; 
while R[i] A NULL do 
o' = Rit}; Rit] = NULL; o = LINK(g, o’); i++ 
endwhile; 
Rit] = o. 


To analyze the amortized cost for deleting the minimum, 
let D(n) be the maximum possible degree of any node 
in a Fibonacci heap of n nodes. The number of linking 
operations in Step 3 is the number of roots we start with, 
which is less than r;_1 + D(n), minus the number of roots 
we end up with, which is r;. After Step 3, all roots have 
different degrees, which implies r; < D(n)+1. It follows 
that the actual cost for the four steps is 


@ < 1414 bei +D(n)-—ri)+(D(n)+1) 
= 3+ 2D(n) + ri-1 ps 


The potential change is ®; — ®;_; = ri —ri—1. The amor- 
tized cost is therefore a; = c; + ®; — ®i—ı < 2D(n) +3. 
We will prove next time that the maximum possible de- 
gree is at most logarithmic in the size of the Fibonacci 
heap, D(n) < 2log,(n + 1). This implies that deleting 
the minimum has logarithmic amortized cost. 


Decreasekey and delete. Besides deletemin, we also 
have operations that delete an arbitrary item and that de- 
crease the key of an item. Both change the structure of 
the heap-ordered trees and are the reason why a Fibonacci 
heap is not a collection of binomial trees but of more gen- 
eral heap-ordered trees. The decreasekey operation re- 
places the item with key x stored in the node v by x — A, 
where A > 0. We will see that this can be done more effi- 
ciently than to delete x and to insert x — A. We decrease 
the key in four steps. 


Step 1. Unlink the tree rooted at v. 
Step 2. Decrease the key in v by A. 


Step 3. Add v to the root cycle and possibly update 
the pointer to the minimum key. 


Step 4. Docascading cuts. 


We will explain cascading cuts shortly, after explaining 
the four steps we take to delete a node v. Before we delete 
a node v, we check whether v = min, and if it is then we 
delete the minimum as explained above. Assume therefore 
that v A min. 


Step 1. Unlink the tree rooted at v. 

Step 2. Merge the root-cycle with the cycle of v’s chil- 
dren. 

Step 3. Dispose of v. 

Step 4. Docascading cuts. 


Figure 48 illustrates the effect of decreasing a key and of 
deleting a node. Both operations create trees that are not 
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Figure 48: A Fibonacci heap initially consisting of three bino- 
mial trees modified by a decreasekey and a delete operation. 


binomial, and we use cascading cuts to make sure that the 
shapes of these trees are not very different from the shapes 
of binomial trees. 


Cascading cuts. Let v be a node that becomes the child 
of another node at time t. We mark v when it loses its first 
child after time t. Then we unmark v, unlink it, and add it 
to the root-cycle when it loses its second child thereafter. 
We call this operation a cut, and it may cascade because 
one cut can cause another, and so on. Figure 49 illus- 
trates the effect of cascading in a heap-ordered tree with 
two marked nodes. The first step decreases key 10 to 7, 
and the second step cuts first node 5 and then node 4. 
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Figure 49: The effect of cascading after decreasing 10 to 7. 
Marked nodes are shaded. 


Summary analysis. As mentioned earlier, we will prove 
D(n) < 2log,(n+1) next time. Assuming this bound, we 
are able to compute the amortized cost of all operations. 
The actual cost of Step 4 in decreasekey or in delete is the 
number of cuts, c;. The potential changes because there 
are c; new roots and c; fewer marked nodes. Also, the last 
cut may introduce a new mark. Thus 


®; — ii = Ti — Tii + 2m; — 2My4-1 
S Ci — 2c; +2 
= —G +2. 


The amortized cost is therefore a; = c; + ®; — ®;j—ı < 
ci — (2 — ci) = 2. The first three steps of a decreasekey 
operation take only a constant amount of actual time and 
increase the potential by at most a constant amount. It 
follows that the amortized cost of decreasekey, including 
the cascading cuts in Step 4, is only a constant. Similarly, 
the actual cost of a delete operation is at most a constant, 
but Step 2 may increase the potential of the Fibonacci heap 
by as much as D(n). The rest is bounded from above by 
a constant, which implies that the amortized cost of the 
delete operation is O(log n). We summarize the amortized 
cost of the various operations supported by the Fibonacci 
heap: 


find the minimum Od) 
meld two heaps Od) 
insert a new item Od) 
delete the minimum O(log n) 
decrease the key of a node Od) 
delete a node Odog n) 


We will later see graph problems for which the difference 
in the amortized cost of the decreasekey and delete op- 
erations implies a significant improvement in the running 
time. 


12 Solving Recurrence Relations 


Recurrence relations are perhaps the most important tool 
in the analysis of algorithms. We have encountered sev- 
eral methods that can sometimes be used to solve such 
relations, such as guessing the solution and proving it by 
induction, or developing the relation into a sum for which 
we find a closed form expression. We now describe a new 
method to solve recurrence relations and use it to settle 
the remaining open question in the analysis of Fibonacci 
heaps. 


Annihilation of sequences. Suppose we are given an in- 
finite sequence of numbers, A = (do, a1, @2,...). We can 
multiply with a constant, shift to the left and add another 
sequence: 


kA = (kao, kay, kaa, oe Ns 
LA = (a1, 42, 43,...), 
A+B = (ao + bo, a1 + b1, 2 + bo,...). 


As an example, consider the sequence of powers of two, 
a; = 2'. Multiplying with 2 and shifting to the left give 
the same result. Therefore, 
LA—2A = (0,0,0,...). 

We write LA — 2A = (L — 2)A and think of L — 2 as an 
operator that annihilates the sequence of powers of 2. In 
general, L — k annihilates any sequence of the form (ck’). 
What does L — k do to other sequences A = (clt), when 
LARK? 


(L—k)A (cl, cl, cl?,...) — (ck, cké, cke’,...) 
= (€—k)lc, ct, cl’,...) 


(€—k)A. 


We see that the operator L — k annihilates only one type 
of sequence and multiplies other similar sequences by a 
constant. 


Multiple operators. Instead of just one, we can ap- 
ply several operators to a sequence. We may multiply 
with two constants, k(/A) = (ké)A, multiply and shift, 
L(kA) = k(LA), and shift twice, L(LA) = L?A. For 
example, (L — k)(L — £) annihilates all sequences of the 
form (ckt + dé’), where we assume k # £. Indeed, L — k 
annihilates (ck*) and leaves behind ((¢ — k)dé'), which is 
annihilated by L — £. Furthermore, (L — k)(L — £) anni- 
hilates no other sequences. More generally, we have 
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Fact. (L — kı)(L — k2)... (L — kn) annihilates all se- 
quences of the form (ck + coks +...+ nk). 


What if k = £? To answer this question, we consider 
(L — k)? lik’) (L—k)(G+ 1) ot! — ak 

= (L-EN) 

= (0). 


More generally, we have 


Fact. (L — k)” annihilates all sequences of the form 
(p(i)k*), with p(z) a polynomial of degree n — 1. 


Since operators annihilate only certain types of sequences, 
we can determine the sequence if we know the annihilating 
operator. The general method works in five steps: 


. Write down the annihilator for the recurrence. 
. Factor the annihilator. 


1 
2 
3. Determine what sequence each factor annihilates. 
4. Put the sequences together. 

5 


. Solve for the constants of the solution by using initial 
conditions. 


Fibonaccinumbers. We put the method to a test by con- 
sidering the Fibonacci numbers defined recursively as fol- 
lows: 


Fy = 0, 
F = 1, 
F; = Fy-4 + Fj- for j > 2. 


Writing a few of the initial numbers, we get the sequence 
(0,1,1,2,3,5,8,...). We notice that L? — L — 1 annihi- 
lates the sequence because 

(L? -L-—1)(F;) = L°(F;)-— L(F;)— (F3) 
= (Fj+2) — (Fj+1) — (F3) 
= (0). 


If we factor the operator into its roots, we get 


L’ -L-1 = (L-¢)(L-9), 
where 
1 5 
Y = ast = 1.618..., 
1— v5 
y = l-ọ = a = — 0.618... 


The first root is known as the golden ratio because it repre- 
sents the aspect ratio of a rectangular piece of paper from 
which we may remove a square to leave a smaller rect- 
angular piece of the same ratio: y : 1 = 1: 9-1. 
Thus we know that (L — y)(L — p) annihilates (F;) and 
this means that the j-th Fibonacci number is of the form 
Fj = cp! +t. We get the constant factors from the 
initial conditions: 


Fo 
Fi 


G EE, 
cy + CP. 


Solving the two linear equations in two unknowns, we get 
c= 1/v5 and € = —1/v5. This implies that 


1 (1+v5\’ 1 (1-v5\" 
v5\ 2 v5\ 2 l 
From this viewpoint, it seems surprising that F} turns out 
to be an integer for all j. Note that |p| > 1 and |p| < 1. 
It follows that for growing exponent j, yÍ goes to infinity 
and y’ goes to zero. This implies that F is approximately 


y /\/5, and that this approximation becomes more and 
more accurate as j grows. 


j = 


Maximum degree. Recall that D(n) is the maximum 
possible degree of any one node in a Fibonacci heap of 
size n. We need two easy facts about the kind of trees that 
arise in Fibonacci heaps in order to show that D(n) is at 
most logarithmic in n. Let v be a node of degree j, and 
let 41, H2,- . - , Hj be its children ordered by the time they 
were linked to v. 


DEGREE LEMMA. The degree of u; is at least i — 2. 


PROOF. Recall that nodes are linked only during the 
deletemin operation. Right before the linking happens, the 
two nodes are roots and have the same degree. It follows 
that the degree of u; was at least ¿ — 1 at the time it was 
linked to v. The degree of u; might have been even higher 
because it is possible that v lost some of the older children 
after u; had been linked. After being linked, u; may have 
lost at most one of its children, for else it would have been 
cut. Its degree is therefore at least 2 — 2, as claimed. 


SIZE LEMMA. The number of descendents of v (includ- 
ing v) is at least Fy+2. 


PROOF. Let s; be the minimum number of descendents a 
node of degree 7 can have. We have so = 1 and sı = 2. 
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For larger j, we get sj from sj—ı by adding the size of a 
minimum tree with root degree j —2, which is s;_2. Hence 
Sj = Sj—1 + Sj—2, which is the same recurrence relation 
that defines the Fibonacci numbers. The initial values are 
shifted two positions so we get s; = Fj+2, as claimed. 


Consider a Fibonacci heap with n nodes and let v be a 
node with maximum degree D = D(n). The Size Lemma 
implies n > Fp+2. The Fibonacci number with index 
D + 2 is roughly pP +? //5. Because G?+? < v5, we 
have 


After rearranging the terms and taking the logarithm to the 
base y, we get 


D < log V5(n+1)—2. 


Recall that log, £ = log, x/ log, y and use the calculator 

to verify that log, p = 0.694... > 0.5 and log, v5 = 

1.672... < 2. Hence 

loga(n + 1) 
logs 9 

< 2loga(n +1). 


D < +log, V5- 2 


Non-homogeneous terms. We now return to the anni- 
hilation method for solving recurrence relations and con- 
sider 
aj = @Qj-1 + aj—2 +1. 
This is similar to the recurrence that defines Fibonacci 
numbers and describes the minimum number of nodes in 
an AVL tree, also known as height-balanced tree. It is de- 
fined by the requirement that the height of the two sub- 
trees of a node differ by at most 1. The smallest tree 
of height j thus consists of the root, a subtree of height 
j — 1 and another subtree of height 7 — 2. We refer to the 
terms involving a; as the homogeneous terms of the re- 
lation and the others as the non-homogeneous terms. We 
know that L? — L — 1 annihilates the homogeneous part, 
aj = aj—1 + aj—2. If we apply it to the entire relation we 
get 
(L? — L — 1) (a5) (aj+2) — (aj+1) — (aj) 

(ljas): 

The remaining sequence of Is is annihilated by L — 1. 


In other words, (L — y)(L — )(L — 1) annihilates (a;) 
implying that a; = cp? + P@! + c’1J. It remains to find 


the constants, which we get from the boundary conditions 
ag = 1, ay = 2 and a2 = 4; 


G oF T + d = 1, 
ye + PT + d = 2, 
ge + Pe + d = 4 


Noting that y? = ọ + 1, P? = F + 1, and ọ - Y = V5 
we get c = (5 + 2V5)/5, € = (5 — 2/5) /5, and c! = —1. 
The minimum number of nodes of a height-7 AVL tree is 
therefore roughly the constant c times y. Conversely, the 
maximum height of an AVL tree with n = cyf nodes is 
roughly j = log, (n/c) = 1.440...-logyn + O(1). In 
words, the height-balancing condition implies logarithmic 
height. 


Transformations. We extend the set of recurrences we 
can solve by employing transformations that produce rela- 
tions amenable to the annihilation method. We demon- 
strate this by considering mergesort, which is another 
divide-and-conquer algorithm that can be used to sort a 
list of n items: 


Step 1. Recursively sort the left half of the list. 
Step 2. Recursively sort the right half of the list. 


Step 3. Merge the two sorted lists by simultaneously 
scanning both from beginning to end. 


The running time is described by the solution to the recur- 
rence 


TO) = 1, 
T(n) = 2T(n/2)+n. 


We have no way to work with terms like T(n/2) yet. 
However, we can transform the recurrence into a more 
manageable form. Defining n = 2' and t; = T(2') we 
get 


to = 1, 
t = 2tj-1+2°. 


The homogeneous part is annihilated by L — 2. Similarly, 
non-homogeneous part is annihilated by L — 2. Hence, 
(L — 2)? annihilates the entire relation and we get t; = 
(ci +@)2". Expressed in the original notation we thus have 
T(n) = (cloggn + @)n = O(nlogn). This result is of 
course no surprise and reconfirms what we learned earlier 
about sorting. 
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The Master Theorem. Itis sometimes more convenient 
to look up the solution to a recurrence relation than play- 
ing with different techniques to see whether any one can 
make it to yield. Such a cookbook method for recurrence 
relations of the form 
T(n) = aT(n/b) + f(n) 

is provided by the following theorem. Here we assume 
that a > 1 and b > 1 are constants and that f is a well- 
behaved positive function. 


MASTER THEOREM. Define c = log, a and let £ be an 
arbitrarily small positive constant. Then 


O(n°) if f(n) = O(n*~*), 
T(n) = O(n°logn) if f(n) = O(n‘), 
O(f(n)) if f(n) = (n°5). 


The last of the three cases also requires a usually satis- 
fied technical condition, namely that af(n/b) < df(n) 
for some constant 6 strictly less than 1. For example, this 
condition is satisfied in T (n) = 2T (n/2) + n? which im- 
plies T(n) = O(n?). 


As another example consider the relation T(n) = 
2T(n/2) + n that describes the running time of merge- 
sort. We have c = log, 2 = 1 and f(n) = n = O(n°). 
The middle case of the Master Theorem applies and we 
get T(n) = O(n logn), as before. 


Third Homework Assignment 


Write the solution to each problem on a single page. The 
deadline for handing in solutions is October 14. 


Problem 1. (20 = 10 + 10 points). Consider a lazy ver- 
sion of heapsort in which each item in the heap is 
either smaller than or equal to every other item in its 
subtree, or the item is identified as uncertified. To 
certify an item, we certify its children and then ex- 
change it with the smaller child provided it is smaller 
than the item itself. Suppose A[1..n] is a lazy heap 
with all items uncertified. 


(a) How much time does it take to certify A[1]? 

(b) Does certifying A[1] turn A into a proper heap 
in which every item satisfies the heap property? 
(Justify your answer.) 


Problem 2. (20 points). Recall that Fibonacci numbers 
are defined recursively as fo = 0, fF, = 1, and Fn = 
Fy-1+Fn—2. Prove the square of the n-th Fibonacci 
number differs from the product of the two adjacent 
numbers by one: F? = Fy-1- Faga + (-1)"*1. 


Problem 3. (20 points). Professor Pinocchio claims that 
the height of an n-node Fibonacci heap is at most 
some constant times log, n. Show that the Profes- 
sor is mistaken by exhibiting, for any integer n, a 
sequence of operations that create a Fibonacci heap 
consisting of just one tree that is a linear chain of n 
nodes. 


Problem 4. (20 = 10+ 10 points). To search in a sorted 
array takes time logarithmic in the size of the array, 
but to insert a new items takes linear time. We can 
improve the running time for insertions by storing the 
items in several instead of just one sorted arrays. Let 
n be the number of items, let k = [log.(n+1)], 
and write n = Nnk—1ınNk—2... No in binary notation. 
We use k sorted arrays A; (some possibly empty), 
where A; stores n;2Ż items. Each item is stored ex- 
actly once, and the total size of the arrays is indeed 
Pa ni2? = n. Although each individual array is 
sorted, there is no particular relationship between the 
items in different arrays. 


(a) Explain how to search in this data structure and 
analyze your algorithm. 
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(b) Explain how to insert a new item into the data 
structure and analyze your algorithm, both in 
worst-case and in amortized time. 


Problem 5. (20 = 10 + 10 points). Consider a full bi- 
nary tree with n leaves. The size of a node, s(v), is 
the number of leaves in its subtree and the rank is 
the floor of the binary logarithm of the size, r(v) = 


[logs s(v)]. 


(a) Is it true that every internal node v has a child 
whose rank is strictly less than the rank of v? 

(b) Prove that there exists a leaf whose depth 
(length of path to the root) is at most logs n. 


IV GRAPH ALGORITHMS 


Graph Search 

Shortest Paths 

Minimum Spanning Trees 
Union-find 

Fourth Homework Assignment 
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13 Graph Search 


We can think of graphs as generalizations of trees: they 
consist of nodes and edges connecting nodes. The main 
difference is that graphs do not in general represent hier- 
archical organizations. 


Types of graphs. Different applications require differ- 
ent types of graphs. The most basic type is the simple 
undirected graph that consists of a set V of vertices and a 
set E of edges. Each edge is an unordered pair (a set) of 
two vertices. We always assume V is finite, and we write 


Figure 50: A simple undirected graph with vertices 0, 1,2, 3,4 
and edges {0, 1}, {1, 2}, {2, 3}, {3, 0}, {3, 4}. 


(5) for the collection of all unordered pairs. Hence F is a 
subset of CG) Note that because F is a set, each edge can 
occur only once. Similarly, because each edge is a set (of 
two vertices), it cannot connect to the same vertex twice. 
Vertices u and v are adjacent if {u,v} € E. In this case u 


and v are called neighbors. Other types of graphs are 


directed: ECVxY. 

weighted: has a weighting function w : E — R. 
labeled: has a labeling function £ : V — Z. 
non-simple: there are loops and multi-edges. 


A loop is like an edge, except that it connects to the same 
vertex twice. A multi-edge consists of two or more edges 
connecting the same two vertices. 


Representation. The two most popular data structures 
for graphs are direct representations of adjacency. Let 
V = {0,1,...,n — 1} be the set of vertices. The ad- 
jacency matrix is the n-by-n matrix A = (aij) with 


a a [1 iffistek, 
a 0 if {i,j} Z E. 
For undirected graphs, we have aij = aji, so A is sym- 


metric. For weighted graphs, we encode more informa- 
tion than just the existence of an edge and define a;j as 
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the weight of the edge connecting 7 and j. The adjacency 
matrix of the graph in Figure 50 is 


D 

lI 
orForF © 
GoOOrOrF 
Orroroo 
rFOrROHK 
Orocece 


which is symmetric. Irrespective of the number of edges, 


y 
0 
y y 
L2 | 


y 
[3 | 
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Figure 51: The adjacency list representation of the graph in Fig- 
ure 50. Each edge is represented twice, once for each endpoint. 


the adjacency matrix has n? elements and thus requires a 
quadratic amount of space. Often, the number of edges 
is quite small, maybe not much larger than the number of 
vertices. In these cases, the adjacency matrix wastes mem- 
ory, and a better choice is a sparse matrix representation 
referred to as adjacency lists, which is illustrated in Fig- 
ure 51. It consists of a linear array V for the vertices and 
a list of neighbors for each vertex. For most algorithms, 
we assume that vertices and edges are stored in structures 
containing a small number of fields: 


struct Vertex {int d, f, m; Edge «adj}; 
struct Edge {int v; Edge «next}. 


The d, f, m fields will be used to store auxiliary informa- 
tion used or created by the algorithms. 


Depth-first search. Since graphs are generally not or- 
dered, there are many sequences in which the vertices can 
be visited. In fact, itis not entirely straightforward to make 
sure that each vertex is visited once and only once. A use- 
ful method is depth-first search. It uses a global variable, 
time, which is incremented and used to leave time-stamps 
behind to avoid repeated visits. 


void VISIT(int i) 
1 time++; V{i].d = time; 
forall outgoing edges 77 do 


2 if V[j].d = 0 then 
3 V[j]-m = i; VISIT(J) 
endif 
endfor; 


4 time++; V|i].f = time. 


The test in line 2 checks whether the neighbor j of 7 has 
already been visited. The assignment in line 3 records that 
the vertex is visited from vertex 7. A vertex is first stamped 
in line 1 with the time at which it is encountered. A vertex 
is second stamped in line 4 with the time at which its visit 
has been completed. To prepare the search, we initialize 
the global time variable to 0, label all vertices as not yet 
visited, and call VISIT for all yet unvisited vertices. 


time = 0; 
forall vertices i do V[i].d = 0 endfor; 
forall vertices 7 do 

if Vẹ[i].d = 0 then V[i].m = 0; VISIT(i) endif 
endfor. 


Let n be the number of vertices and m the number of edges 
in the graph. Depth-first search visits every vertex once 
and examines every edge twice, once for each endpoint. 
The running time is therefore O(n + m), which is propor- 
tional to the size of the graph and therefore optimal. 


DFS forest. Figure 52 illustrates depth-first search by 
showing the time-stamps d and f and the pointers m in- 
dicating the predecessors in the traversal. We call an edge 
{i,j} € E a tree edge if i = V|j].a or j = V{i].7 anda 
back edge, otherwise. The tree edges form the DFS forest 


Figure 52: The traversal starts at the vertex with time-stamp 1. 
Each node is stamped twice, once when it is first encountered 
and another time when its visit is complete. 


of the graph. The forest is a tree if the graph is connected 
and a collection of two or more trees if it is not connected. 
Figure 53 shows the DFS forest of the graph in Figure 52 
which, in this case, consists of a single tree. The time- 
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Figure 53: Tree edges are solid and back edges are dotted. 
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stamps d are consistent with the preorder traversal of the 
DFS forest. The time-stamps f are consistent with the 
postorder traversal. The two stamps can be used to decide, 
in constant time, whether two nodes in the forest live in 
different subtrees or one is a descendent of the other. 


NESTING LEMMA. Vertex 7 is a proper descendent of 
vertex i in the DFS forest iff V[i].d < V[j].d as well 
as V[j].f < V[i].f. 


Similarly, if you have a tree and the preorder and postorder 
numbers of the nodes, you can determine the relation be- 
tween any two nodes in constant time. 


Directed graphs and relations. As mentioned earlier, 
we have a directed graph if all edges are directed. A 
directed graph is a way to think and talk about a mathe- 
matical relation. A typical problem where relations arise 
is scheduling. Some tasks are in a definite order while 
others are unrelated. An example is the scheduling of 
undergraduate computer science courses, as illustrated in 
Figure 54. Abstractly, a relation is a pair (V, E), where 


Comput. Org. Operating Distributed 
and Programm. Systems Inform. Syst. 
Program Design Program Design 
and AnalysisI and Analysis IT 104 = 110 = 212 
006 > 100 AS Ne Ye 
108 -———— 214 


Software Design 
and Implementation 


Comput. Networks 
and Distr. Syst. 


Figure 54: A subgraph of the CPS course offering. The courses 
CPS 104 and CPS108 are incomparable, CPS 104 is a predecessor 
of CPS110, and so on. 


v 
E 


{0,1,...,n — 1} is a finite set of elements and 


C V x V is a finite set of ordered pairs. Instead of 


(i,j) € E we write i < j and instead of (V, E) we write 
(V, <). If i < j then 7 is a predecessor of j and j is a suc- 
cessor of i. The terms relation, directed graph, digraph, 
and network are all synonymous. 


Directed acyclic graphs. A cycle in a relation is a se- 
quence to < 2% < < ik < to. Even io < to 
is a cycle. A linear extension of (V, <) is an ordering 
Jo, J1,- --, Jn—1 Of the elements that is consistent with the 
relation. Formally this means that jẹ < je implies k < £. 
A directed graph without cycle is a directed acyclic graph. 


EXTENSION LEMMA. (V, <) has a linear extension iff it 
contains no cycle. 


PROOF. “==>” is obvious. We prove “<—” by induction. 
A vertex s € V is called a source if it has no predecessor. 
Assuming (V, <) has no cycle, we can prove that V has 
a source by following edges against their direction. If we 
return to a vertex that has already been visited, we have 
a cycle and thus a contradiction. Otherwise we get stuck 
at a vertex s, which can only happen because s has no 
predecessor, which means s is a source. 


Let U = V—{s} and note that (U, <) is a relation that is 
smaller than (V, <). Hence (U, <) has a linear extension 
by induction hypothesis. Call this extension X and note 
that s, X is a linear extension of (V, <). 


Topological sorting with queue. The problem of con- 
structing a linear extension is called topological sorting. 
A natural and fast algorithm follows the idea of the proof: 
find a source s, print s, remove s, and repeat. To expedite 
the first step of finding a source, each vertex maintains 
its number of predecessors and a queue stores all sources. 
First, we initialize this information. 


forall vertices j do V[j].d = 0 endfor; 
forall vertices 7 do 

forall successors j of i do V[j].d++ endfor 
endfor; 
forall vertices j do 

if V[j].d = 0 then ENQUEUE(j) endif 
endfor. 


Next, we compute the linear extension by repeated dele- 
tion of a source. 
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while queue is non-empty do 
s = DEQUEUE; 
forall successors j of s do 
V[j].d--; 
if V|j].d = 0 then ENQUEUE(j) endif 
endfor 
endwhile. 


The running time is linear in the number of vertices and 
edges, namely O(n + m). What happens if there is a cycle 
in the digraph? We illustrate the above algorithm for the 
directed acyclic graph in Figure 55. The sequence of ver- 


3,2, 1,0 


Figure 55: The numbers next to each vertex count the predeces- 
sors, which decreases during the algorithm. 


tices added to the queue is also the linear extension com- 
puted by the algorithm. If the process starts at vertex a 
and if the successors of a vertex are ordered by name then 
we get a, f, d, g, c, h, b, e, which we can check is indeed a 
linear extension of the relation. 


Topological sorting with DFS. Another algorithm that 
can be used for topological sorting is depth-first search. 
We output a vertex when its visit has been completed, that 
is, when all its successors and their successors and so on 
have already been printed. The linear extension is there- 
fore generated from back to front. Figure 56 shows the 


Figure 56: The numbers next to each vertex are the two time 
stamps applied by the depth-first search algorithm. The first 
number gives the time the vertex is encountered, and the second 
when the visit has been completed. 


same digraph as Figure 55 and labels vertices with time 


stamps. Consider the sequence of vertices in the order of 
decreasing second time stamp: 


a(16), f(14), g(13), h(12), d(9), c(8), e(7), b(5). 


Although this sequence is different from the one computed 
by the earlier algorithm, it is also a linear extension of the 
relation. 
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14 Shortest Paths 


One of the most common operations in graphs is finding 
shortest paths between vertices. This section discusses 
three algorithms for this problem: breadth-first search 
for unweighted graphs, Dijkstra’s algorithm for weighted 
graphs, and the Floyd-Warshall algorithm for computing 
distances between all pairs of vertices. 


Breadth-first search. We call a graph connected if there 
is a path between every pair of vertices. A (connected) 
component is a maximal connected subgraph. Breadth- 
first search, or BFS, is a way to search a graph. It is sim- 
ilar to depth-first search, but while DFS goes as deep as 
quickly as possible, BFS is more cautious and explores a 
broad neighborhood before venturing deeper. The starting 
point is a vertex s. An example is shown in Figure 57. As 


Figure 57: A sample graph with eight vertices and ten edges 
labeled by breath-first search. The label increases from a vertex 
to its successors in the search. 


before, we call and edge a tree edge if it is traversed by the 
algorithm. The tree edges define the BFS tree, which we 
can use to redraw the graph in a hierarchical manner, as in 
Figure 58. In the case of an undirected graph, no non-tree 
edge can connect a vertex to an ancestor in the BFS tree. 
Why? We use a queue to turn the idea into an algorithm. 


Figure 58: The tree edges in the redrawing of the graph in Figure 
57 are solid, and the non-tree edges are dotted. 


First, the graph and the queue are initialized. 


forall vertices i do V[i].d = —1 endfor; 
V[s].d = 0; 


MAKEQUEUE; ENQUEUE(s); SEARCH. 
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A vertex is processed by adding its unvisited neighbors to 
the queue. They will be processed in turn. 


void SEARCH 
while queue is non-empty do 
i = DEQUEUE; 
forall neighbors j of i do 
if V|j].d = —1 then 
V[j].d = Vii].d+1; V[j].T = i; 
ENQUEUE(j) 
endif 
endfor 
endwhile. 


The label V [i].d assigned to vertex i during the traversal is 
the minimum number of edges of any path from s to i. In 
other words, V |i].d is the length of the shortest path from 
s to 7. The running time of BFS for a graph with n vertices 
and m edges is O(n + m). 


Single-source shortest path. BFS can be used to find 
shortest paths in unweighted graphs. We now extend the 
algorithm to weighted graphs. Assume V and E are the 
sets of vertices and edges of a simple, undirected graph 
with a positive weighting function w : E — R+. The 
length or weight of a path is the sum of the weights of 
its edges. The distance between two vertices is the length 
of the shortest path connecting them. For a given source 
s € V, we study the problem of finding the distances and 
shortest paths to all other vertices. Figure 59 illustrates the 
problem by showing the shortest paths to the source s. In 


Figure 59: The bold edges form shortest paths and together the 
shortest path tree with root s. It differs by one edge from the 
breadth-first tree shown in Figure 57. 


the non-degenerate case, in which no two paths have the 
same length, the union of all shortest paths to s is a tree, 
referred to as the shortest path tree. In the degenerate case, 
we can break ties such that the union of paths is a tree. 


As before, we grow a tree starting from s. Instead of a 
queue, we use a priority queue to determine the next vertex 
to be added to the tree. It stores all vertices not yet in the 


tree and uses V[i|.d for the priority of vertex i. First, we 
initialize the graph and the priority queue. 


V[s].d = 0; V[s].7 = —1; INSERT(s); 
forall vertices i Æ s do 

V[i].d = co; INSERT(?) 
endfor. 


After initialization the priority queue stores s with priority 
0 and all other vertices with priority oo. 


Dijkstra’s algorithm. We mark vertices in the tree to 
distinguish them from vertices that are not yet in the tree. 
The priority queue stores all unmarked vertices 2 with pri- 
ority equal to the length of the shortest path that goes from 
i in one edge to a marked vertex and then to s using only 
marked vertices. 


while priority queue is non-empty do 
i = EXTRACTMIN; mark 7; 
forall neighbors j of i do 
if j is unmarked then 
V[j].d = min{w(i7) + V [i].d, V[y].d} 
endif 
endfor 
endwhile. 


Table 3 illustrates the algorithm by showing the informa- 
tion in the priority queue after each iteration of the while- 
loop operating on the graph in Figure 59. The mark- 


s 
a 
b 
c 
d 
e 
f 
g 


Table 3: Each column shows the contents of the priority queue. 
Time progresses from left to right. 


ing mechanism is not necessary but clarifies the process. 
The algorithm performs n EXTRACTMIN operations and 
at most m DECREASEKEY operations. We compare the 
running time under three different data structures used to 
represent the priority queue. The first is a linear array, as 
originally proposed by Dijkstra, the second is a heap, and 
the third is a Fibonacci heap. The results are shown in 
Table 4. We get the best result with Fibonacci heaps for 
which the total running time is O(n logn + m). 
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[ry | heap | Fiep 


E 
EXTRACTMINS n nlogn nlogn 
DECREASEKEYs m mlogm m 
Table 4: Running time of Dijkstra’s algorithm for three different 


implementations of the priority queue holding the yet unmarked 
vertices. 


Correctness. Itis not entirely obvious that Dijkstra’s al- 
gorithm indeed finds the shortest paths to s. To show that 
it does, we inductively prove that it maintains the follow- 
ing two invariants. 


(A) For every unmarked vertex j, V[j].d is the length of 
the shortest path from j to s that uses only marked 
vertices other than 7. 


(B) For every marked vertex i, V [i].d is the length of the 
shortest path from 2 to s. 


PROOF. Invariant (A) is true at the beginning of Dijkstra’s 
algorithm. To show that it is maintained throughout the 
process, we need to make sure that shortest paths are com- 
puted correctly. Specifically, if we assume Invariant (B) 
for vertex 7 then the algorithm correctly updates the prior- 
ities V[j].d of all neighbors j of i, and no other priorities 
change. 


Figure 60: The vertex y is the last unmarked vertex on the hypo- 
thetically shortest, dashed path that connects 7 to s. 


At the moment vertex i is marked, it minimizes V [j].d 
over all unmarked vertices j. Suppose that, at this mo- 
ment, V [i].d is not the length of the shortest path from i to 
s. Because of Invariant (A), there is at least one other un- 
marked vertex on the shortest path. Let the last such vertex 
be y, as shown in Figure 60. But then V[y].d < V{i].d, 
which is a contradiction to the choice of 7. 


We used (B) to prove (A) and (A) to prove (B). To make 
sure we did not create a circular argument, we parametrize 
the two invariants with the number k of vertices that are 


marked and thus belong to the currently constructed por- 
tion of the shortest path tree. To prove (Ax) we need (Bẹ) 
and to prove (B) we need (Ax_1). Think of the two in- 
variants as two recursive functions, and for each pair of 
calls, the parameter decreases by one and thus eventually 
becomes zero, which is when the argument arrives at the 
base case. 


All-pairs shortest paths. We can run Dijkstra’s algo- 
rithm n times, once for each vertex as the source, and thus 
get the distance between every pair of vertices. The run- 
ning time is O(n? log n + nm) which, for dense graphs, is 
the same as O(n). Cubic running time can be achieved 
with a much simpler algorithm using the adjacency matrix 
to store distances. The idea is to iterate n times, and after 
the k-th iteration, the computed distance between vertices 
i and j is the length of the shortest path from 7 to j that, 
other than 7 and j, contains only vertices of index k or less. 


fork=l1tondo 
fori=ltondo 
forj =ltondo 
Ali, j] = min{ Ali, j], Ali, k] + Afk, j1} 
endfor 
endfor 
endfor. 


The only information needed to update Afi, j] during the 
k-th iteration of the outer for-loop are its old value and 
values in the k-th row and the k-th column of the prior 
adjacency matrix. This row remains unchanged in this it- 
eration and so does this column. We therefore do not have 
to use two arrays, writing the new values right into the old 
matrix. We illustrate the algorithm by showing the adja- 
cency, or distance matrix before the algorithm in Figure 
61 and after one iteration in Figure 62. 


sabc def g 


s|0O 5 10 4 5 
a|5 0 4 5 10 
b |10 4 0 
c |4 
d |5 10 
e 5 
f 10 
10 0 


Figure 61: Adjacency, or distance matrix of the graph in Figure 
57. All blank entries store oo. 


52 


0;5 10 4 5 s |0|5 a 4 5 OS 
5/0 4 {9{|10)5 10 a | 0 )4 9 10 3 10 
10| 4 0 | 14) 15 b |9| 4] 0] 13] 14) 9 | 14 
4/914) 0 c |4/9]13)0 4/14/19 
5) 10] 15} 4 10) d | 5 | 10) 14) 4 0 | 15) 20} 10 
5 0 6 e |10)/5|91|14)15;)0 6 
10 6 0 f | 15} 10] 14) 19} 20} 6 0 
10 0j £ 10 0 
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Figure 62: Matrix after each iteration. The k-th row and colum 
are shaded and the new, improved distances are high-lighted. 


The algorithm works for weighted undirected as well 
as for weighted directed graphs. Its correctness is easily 
verified inductively. The running time is O(n?). 


15 Minimum Spanning Trees 


When a graph is connected, we may ask how many edges 
we can delete before it stops being connected. Depending 
on the edges we remove, this may happen sooner or later. 
The slowest strategy is to remove edges until the graph 
becomes a tree. Here we study the somewhat more dif- 
ficult problem of removing edges with a maximum total 
weight. The remaining graph is then a tree with minimum 
total weight. Applications that motivate this question can 
be found in life support systems modeled as graphs or net- 
works, such as telephone, power supply, and sewer sys- 
tems. 


Free trees. An undirected graph (U, T) is a free tree if 
it is connected and contains no cycle. We could impose a 
hierarchy by declaring any one vertex as the root and thus 
obtain a rooted tree. Here, we have no use for a hierarchi- 
cal organization and exclusively deal with free trees. The 
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Figure 63: Adding the edge dg to the tree creates a single cycle 
with vertices d, g, h, f,e, a. 


number of edges of a free tree is always one less than the 
number of vertices. Whenever we add a new edge (con- 
necting two old vertices) we create exactly one cycle. This 
cycle can be destroyed by deleting any one of its edges, 
and we get a new free tree, as in Figure 63. Let (V, E) be 
a connected and undirected graph. A subgraph is another 
graph (U, T) with U C V and T C E. It is a spanning 
tree if it is a free tree with U = V. 


Minimum spanning trees. For the remainder of this 
section, we assume that we also have a weighting func- 
tion, w : E — R. The weight of subgraph is then the 
total weight of its edges, w(T) = X er w(e). A mini- 
mum spanning tree, or MST of G is a spanning tree that 
minimizes the weight. The definitions are illustrated in 
Figure 64 which shows a graph of solid edges with a min- 
imum spanning tree of bold edges. A generic algorithm 
for constructing an MST grows a tree by adding more and 
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Figure 64: The bold edges form a spanning tree of weight 0.9 + 
1.2 + 1.3 + 1.4+ 1.1 + 1.2 + 1.6 + 1.9 = 10.6. 


more edges. Let A C E be a subset of some MST of a 
connected graph (V, E). An edge uv € E — Ais safe for 
A if AU {uv} is also subset of some MST. The generic 
algorithm adds safe edges until it arrives at an MST. 


A=; 

while (V, A) is not a spanning tree do 
find a safe edge uv; A = AU {uv} 

endwhile. 


As long as A is a proper subset of an MST there are safe 
edges. Specifically, if (V, T) is an MST and A C T then 
all edges in T — A are safe for A. The algorithm will 
therefore succeed in constructing an MST. The only thing 
that is not yet clear is how to find safe edges quickly. 


Cuts. To develop a mechanism for identifying safe 
edges, we define a cut, which is a partition of the vertex 
set into two complementary sets, V = W Ù (V — W). Itis 
crossed by an edge uv € E if u € W and v € V — W, and 
it respects an edge set A if A contains no crossing edge. 
The definitions are illustrated in Figure 65. 
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Figure 65: The vertices inside and outside the shaded regions 
form a cut that respects the collection of solid edges. The dotted 
edges cross the cut. 


CUT LEMMA. Let A be subset of an MST and consider a 
cut W Ù (V — W) that respects A. If wv is a crossing 
edge with minimum weight then wv is safe for A. 


PROOF. Consider a minimum spanning tree (V, T) with 
A C T. If uv € T then we are done. Otherwise, let 
T’ = TU {uv}. Because T is a tree, there is a unique 
path from u to vin T. We have u € W and v € V —W, 
so the path switches at least once between the two sets. 
Suppose it switches along xy, as in Figure 66. Edge ry 
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Figure 66: Adding wv creates a cycle and deleting xy destroys 
the cycle. 


crosses the cut, and since A contains no crossing edges we 
have zy ¢ A. Because uv has minimum weight among 
crossing edges we have w(uv) < w(xy). Define T” = 
T’ — {xy}. Then (V, T”) is a spanning tree and because 


w(T”) = w(T) — w(zy) + w(uv) < w(T) 


it is a minimum spanning tree. The claim follows because 
AU {uv} CT”. 


A typical application of the Cut Lemma takes a compo- 
nent of (V, A) and defines W as the set of vertices of that 
component. The complementary set V — W contains all 
other vertices, and crossing edges connect the component 
with its complement. 


Prim’s algorithm. Prim’s algorithm chooses safe edges 
to grow the tree as a single component from an arbitrary 
first vertex s. Similar to Dijkstra’s algorithm, the vertices 
that do not yet belong to the tree are stored in a priority 
queue. For each vertex 7 outside the tree, we define its 
priority V [i].d equal to the minimum weight of any edge 
that connects 7 to a vertex in the tree. If there is no such 
edge then V |i].d = oo. In addition to the priority, we store 
the index of the other endpoint of the minimum weight 
edge. We first initialize this information. 


V[s].d = 0; V[s].7 = —1; INSERT(s); 
forall vertices i Æ s do 

V [i].d = 00; INSERT(i) 
endfor. 
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The main algorithm expands the tree by one edge at a time. 
It uses marks to distinguish vertices in the tree from ver- 
tices outside the tree. 


while priority queue is non-empty do 
i = EXTRACTMIN; mark 7; 
forall neighbors j of i do 
if j is unmarked and w(ij) < V[j].d then 
Vij].d = w(ij); Vij] = i 
endif 
endfor 
endwhile. 


After running the algorithm, the MST can be recovered 
from the 7-fields of the vertices. The algorithm together 


with its initialization phase performs n = |V| insertions 
into the priority queue, n extractmin operations, and at 
most m = |E| decreasekey operations. Using the Fi- 


bonacci heap implementation, we get a running time of 
O(n logn + m), which is the same as for constructing the 
shortest-path tree with Dijkstra’s algorithm. 


Kruskal’s algorithm. Kruskal’s algorithm is another 
implementation of the generic algorithm. It adds edges in 
a sequence of non-decreasing weight. At any moment, the 
chosen edges form a collection of trees. These trees merge 
to form larger and fewer trees, until they eventually com- 
bine into a single tree. The algorithm uses a priority queue 
for the edges and a set system for the vertices. In this 
context, the term ‘system’ is just another word for ‘set’, 
but we will use it exclusively for sets whose elements are 
themselves sets. Implementations of the set system will 
be discussed in the next lecture. Initially, A = Ø, the pri- 
ority queue contains all edges, and the system contains a 
singleton set for each vertex, C = {{u} | u € V}. The 
algorithm finds an edge with minimum weight that con- 
nects two components defined by A. We set W equal to 
the vertex set of one component and use the Cut Lemma 
to show that this edge is safe for A. The edge is added to 
A and the process is repeated. The algorithm halts when 
only one tree is left, which is the case when A contains 
n— 1 = |V] — 1 edges. 


A=9; 
while |A| <<n—1l1do 
uv = EXTRACTMIN; 
find P,Q € C with u € P and v € Q; 
if P #Q then 
A = AU {uv}; merge P and Q 
endif 
endwhile. 


The running time is O(m log m) for the priority queue op- 
erations plus some time for maintaining C. There are two 
operations for the set system, namely finding the set that 
contains a given element, and merging two sets into one. 


An example. We illustrate Kruskal’s algorithm by ap- 
plying it to the weighted graph in Figure 64. The sequence 
of edges sorted by weight is cd, fi, fh, ad, ae, hi, de, ef, 
ac, gh, dg, bf, eg, bi, ab. The evolution of the set system 


Figure 67: Eight union operations merge the nine singleton sets 
into one set. 


is illustrated in Figure 67, and the MST computed with 
Kruskal’s algorithm and indicated with dotted edges is the 
same as in Figure 64. The edges cd, fi, fh, ad, ae are all 
added to the tree. The next two edge, hz and de, are not 
added because they each have both endpoints in the same 
component, and adding either edge would create a cycle. 
Edge ef is added to the tree giving rise to a set in the sys- 
tem that contains all vertices other than g and b. Edge ac 
is not added, gh is added, dg is not, and finally bf is added 
to the tree. At this moment the system consists of a single 
set that contains all vertices of the graph. 


As suggested by Figure 67, the evolution of the con- 
struction can be interpreted as a hierarchical clustering of 
the vertices. The specific method that corresponds to the 
evolution created by Kruskal’s algorithm is referred to as 
single-linkage clustering. 
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16 Union-Find 


In this lecture, we present two data structures for the dis- 
joint set system problem we encountered in the implemen- 
tation of Kruskal’s algorithm for minimum spanning trees. 
An interesting feature of the problem is that m operations 
can be executed in a time that is only ever so slightly more 
than linear in m. 


Abstract data type. A disjoint set system is an abstract 
data type that represents a partition C of a set [n] = 
{1,2,...,n}. In other words, C is a set of pairwise dis- 
joint subsets of [n] such that the union of all sets in C is 
[n]. The data type supports 


set FIND(i): return P € C with i € P; 
void UNION(P,Q):C=C—{P,Q}U{PUQ}. 


In most applications, the sets themselves are irrelevant, 
and it is only important to know when two elements be- 
long to the same set and when they belong to different sets 
in the system. For example, Kruskal’s algorithm executes 
the operations only in the following sequence: 


P = FIND(i); Q = FIND(J); 
if P Æ Q then UNION(P, Q) endif. 


This is similar to many everyday situations where it is usu- 
ally not important to know what it is as long as we recog- 
nize when two are the same and when they are different. 


Linked lists. We construct a fairly simple and reason- 
ably efficient first solution using linked lists for the sets. 
We use a table of length n, and for each 7 € [n], we store 
the name of the set that contains i. Furthermore, we link 
the elements of the same set and use the name of the first 
element as the name of the set. Figure 68 shows a sample 
set system and its representation. It is convenient to also 
store the size of the set with the first element. 


To perform a UNION operation, we need to change the 
name for all elements in one of the two sets. To save time, 
we do this only for the smaller set. To merge the two lists 
without traversing the longer one, we insert the shorter list 
between the first two elements of the longer list. 
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Figure 68: The system consists of three sets, each named by the 
bold element. Each element stores the name of its set, possibly 
the size of its set, and possibly a pointer to the next element in 
the same set. 


void UNION(int P, Q) 
if C|P].size < C[Q].size then P + Q endif; 
C|P].size = C[P].size + C|Q].size; 
second = C|P].next; C[P].next = Q; t = Q; 
whilet#0do 
Clt].set = P; u = t, t = Clt].next 
endwhile; C[u].next = second. 


In the worst case, a single UNION operation takes time 
O(n). The amortized performance is much better because 
we spend time only on the elements of the smaller set. 


WEIGHTED UNION LEMMA. n — 1 UNION operations 
applied to a system of n singleton sets take time 
O(nlogn). 


PROOF. For an element, 7, we consider the cardinality of 
the set that contains it, o (i) = C[FIND(i)].size. Each time 
the name of the set that contains i changes, o(i) at least 
doubles. After changing the name k times, we have o (i) > 
2* and therefore k < logy n. In other words, ¿ can be in 
the smaller set of a UNION operation at most log, n times. 
The claim follows because a UNION operation takes time 
proportional to the cardinality of the smaller set. 


Up-trees. Thinking of names as pointers, the above data 
structure stores each set in a tree of height one. We can 
use more general trees and get more efficient UNION op- 
erations at the expense of slower FIND operations. We 
consider a class of algorithms with the following common- 
alities: 


e each set is a tree and the name of the set is the index 
of the root; 


e FIND traverses a path from a node to the root; 


e UNION links two trees. 


It suffices to store only one pointer per node, namely the 
pointer to the parent. This is why these trees are called 
up-trees. It is convenient to let the root point to itself. 


Figure 69: The UNION operations create a tree by linking the 
root of the first set to the root of the second set. 
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Figure 70: The table stores indices which function as pointers as 
well as names of elements and of sets. The white dot represents 
a pointer to itself. 


Figure 69 shows the up-tree generated by executing the 
following eleven UNION operations on a system of twelve 
singleton sets: 2 U 3, 4U 7, 2U 4, 1U 2, 4U 10, 9U 12, 
12U 2, 8U 11, 8U 2, 5U 6, 6U 1. Figure 70 shows the 
embedding of the tree in a table. UNION takes constant 
time and FIND takes time proportional to the length of the 
path, which can be as large as n — 1. 


Weighted union. The running time of FIND can be im- 
proved by linking smaller to larger trees. This is the ide 
of weighted union again. Assume a field C/i].p for the 
index of the parent (C[?|.p = i if i is a root), and a field 
C|i].size for the number of elements in the tree rooted at i. 
We need the size field only for the roots and we need the 
index to the parent field everywhere except for the roots. 
The FIND and UNION operations can now be implemented 
as follows: 
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int FIND(int i) 
if Cli].p Ai then return FIND(C|i].p) endif; 
return i. 


void UNION(int i, j) 
if Cli].size < Clj].size then i j endif; 
Cli].size = C{i].size + Cly].size; Clj].p = i. 


The size of a subtree increases by at least a factor of 2 from 
anode to its parent. The depth of a node can therefore not 
exceed logy n. It follows that FIND takes at most time 
O(log n). We formulate the result on the height for later 
reference. 


HEIGHT LEMMA. An up-tree created from n singleton 
nodes by n — 1 weighted union operations has height 
at most log, n. 


Path compression. We can further improve the time for 
FIND operations by linking traversed nodes directly to the 
root. This is the idea of path compression. The UNION 
operation is implemented as before and there is only one 
modification in the implementation of the FIND operation: 


int FIND(int i) 


if Cli]._p #4 i then Cfi].p = FIND(C{i].p) endif; 
return C{i].p. 
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Figure 71: The operations and up-trees develop from top to bot- 
tom and within each row from left to right. 


If 7 is not root then the recursion makes it the child of a 
root, which is then returned. If 2 is a root, it returns itself 


because in this case C[?].p = i, by convention. Figure 71 
illustrates the algorithm by executing a sequence of eight 
operations ¿U j, which is short for finding the sets that 
contain 7 and 7, and performing a UNION operation if the 
sets are different. At the beginning, every element forms 
its own one-node tree. With path compression, it is diffi- 
cult to imagine that long paths can develop at all. 


Iterated logarithm. We will prove shortly that the iter- 
ated logarithm is an upper bound on the amortized time 
for a FIND operation. We begin by defining the function 
from its inverse. Let F(0) = 1 and F(i + 1) = 27. We 
have F(1) = 2, F(2) = 22, and F(3) = 2?°. In general, 
F (i) is the tower of i 2s. Table 5 shows the values of F 
for the first six arguments. For 7 < 3, F is very small, but 


Mba nna 4 | 5 | 


[E it | 2] 4 | 16 | 65,536 | 25° | 


Table 5: Values of F. 


for 2 = 5 it already exceeds the number of atoms in our 
universe. Note that the binary logarithm of a tower of 7 2s 
is a tower of i— 1 2s. The iterated logarithm is the number 
of times we can take the binary logarithm before we drop 
down to one or less. In other words, the iterated logarithm 
is the inverse of F, 
log*n = minf{i| F(z) >n} 

= min{i| log, log,...log,n < 1}, 


where the binary logarithm is taken 7 times. As n goes to 
infinity, log* n goes to infinity, but very slowly. 


Levels and groups. The analysis of the path com- 
pression algorithm uses two Census Lemmas discussed 
shortly. Let A1, A2, .. . , Am be a sequence of UNION and 
FIND operations, and let T' be the collection of up-trees 
we get by executing the sequence, but without path com- 
pression. In other words, the FIND operations have no 
influence on the trees. The level A(u) of a node p is its 
height of its subtree in T plus one. 


LEVEL CENSUS LEMMA. There are at most n/2°~! 
nodes at level Z. 


PROOF. We use induction to show that a node at level Z 
has a subtree of at least 2⁄7! nodes. The claim follows 
because subtrees of nodes on the same level are disjoint. 
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Note that if u is a proper descendent of another node 
v at some moment during the execution of the operation 
sequence then u is a proper descendent of v in T. In this 
case A(u) < A(V). 
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Figure 72: A schematic drawing of the tree 7’ between the col- 
umn of level numbers on the left and the column of group num- 
bers on the right. The tree is decomposed into five groups, each 
a sequences of contiguous levels. 


Define the group number of a node jy as the iterated 
logarithm of the level, g(j:) = log* A(w). Because the 
level does not exceed n, we have g(j1) < log” n, for every 
node u in T. The definition of g decomposes an up-tree 
into at most 1 + log* n groups, as illustrated in Figure 72. 
The number of levels in group g is F(g)— F'(g—1), which 
gets large very fast. On the other hand, because levels get 
smaller at an exponential rate, the number of nodes in a 
group is not much larger than the number of nodes in the 
lowest level of that group. 


GROUP CENSUS LEMMA. There are at most 2n/F'(g) 
nodes with group number g. 


PROOF. Each node with group number g has level between 
F(g—1)+1 and F(g). We use the Level Census Lemma 
to bound their number: 


2 n- (1+4+ł4+...) 
D %2- = 2F(g-1) 
é=F(g—1)+1 
— an 
F(g)’ 
as claimed. 
Analysis. The analysis is based on the interplay between 


the up-trees obtained with and without path compression. 


The latter are constructed by the weighted union opera- 
tions and eventually form a single tree, which we denote 
as T. The former can be obtained from the latter by the 
application of path compression. Note that in T, the level 
strictly increases from a node to its parent. Path compres- 
sion preserves this property, so levels also increase when 
we climb a path in the actual up-trees. 


We now show that any sequence of m > n UNION and 
FIND operations on a ground set [n] takes time at most 
O(m log* n) if weighted union and path compression is 
used. We can focus on FIND because each UNION opera- 
tion takes only constant time. For a FIND operation Aj, let 
X; be the set of nodes along the traversed path. The total 
time for executing all FIND operations is proportional to 


r = > lxil. 


For u € X;, let p;(p) be the parent during the execution of 
Ai. We partition X; into the topmost two nodes, the nodes 
just below boundaries between groups, and the rest: 


Y; = {pw © X; | pis root or child of root}, 
Zi, = {u€ Xi- Y; | glu) < g(pilu))}, 
Wi = {ue Xi- Y; | gu) = glp: (H) 


Clearly, |Y;| < 2 and |Z;| < log* n. It remains to bound 
the total size of the W;, w = >°, |W;|. Instead of count- 
ing, for each A;, the nodes in W;, we count, for each node 
jt, the FIND operations A; for which u € W;. In other 
words, we count how often js can change parent until its 
parent has a higher group number than u. Each time u 
changes parent, the new parent has higher level than the 
old parent. If follows that the number of changes is at 
most F'(g()) — F(g(u) — 1). The number of nodes with 
group number g is at most 2n/F'(g) by the Group Census 
Lemma. Hence 


log* n 


2 y 


& FO) 
< 2n-(1+log* n). 


-(F(g) — F(g— 1) 


This implies that 


x < 2m-+mlog* n+ 2n(1 + log* n) 


= O(mlog* n), 
assuming m > n. This is an upper bound on the total time 
it takes to execute m FIND operations. The amortized cost 


per FIND operation is therefore at most O(log* n), which 
for all practical purposes is a constant. 
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Summary. We proved an upper bound on the time 
needed for m > n UNION and FIND operations. The 
bound is more than constant per operation, although for 
all practical purposes it is constant. The log“ n bound can 
be improved to an even smaller function, usually referred 
to as a(n) or the inverse of the Ackermann function, that 
goes to infinity even slower than the iterated logarithm. 
It can also be proved that (under some mild assumptions) 
there is no algorithm that can execute general sequences 
of UNION and FIND operations in amortized time that is 
asymptotically less than a(n). 


Fourth Homework Assignment 


Write the solution to each problem on a single page. The 
deadline for handing in solutions is October 30. 


Problem 1. (20 = 10 + 10 points). Consider a free tree 
and let d(u, v) be the number of edges in the path 
connecting u to v. The diameter of the tree is the 
maximum d(u, v) over all pairs of vertices in the tree. 


(a) Give an efficient algorithm to compute the di- 
ameter of a tree. 
(b) Analyze the running time of your algorithm. 


Problem 2. (20 points). Design an efficient algorithm to 
find a spanning tree for a connected, weighted, undi- 
rected graph such that the weight of the maximum 
weight edge in the spanning tree is minimized. Prove 
the correctness of your algorithm. 


Problem 3. (7 + 6 + 7 points). A weighted graph G = 
(V, E) is a near-tree if it is connected and has at most 
n + 8 edges, where n is the number of vertices. Give 
an O(n)-time algorithm to find a minimum weight 
spanning tree for G. 


Problem 4. (10 + 10 points). Given an undirected 
weighted graph and vertices s, t, design an algorithm 
that computes the number of shortest paths from s to 
t in the case: 


(a) All weights are positive numbers. 
(b) All weights are real numbers. 


Analyze your algorithm for both (a) and (b). 


Problem 5. (20 = 10 + 10 points). The off-line mini- 
mum problem is about maintaining a subset of |n] = 
{1,2,...,m} under the operations INSERT and EX- 
TRACTMIN. Given an interleaved sequence of n in- 
sertions and m min-extractions, the goal is to deter- 
mine which key is returned by which min-extraction. 
We assume that each element i € [n] is inserted ex- 
actly once. Specifically, we wish to fill in an array 
E|1..m] such that [i] is the key returned by the i- 
th min-extraction. Note that the problem is off-line, 
in the sense that we are allowed to process the entire 
sequence of operations before determining any of the 
returned keys. 


(a) Describe how to use a union-find data structure 
to solve the problem efficiently. 


60 


(b) Give a tight bound on the worst-case running 
time of your algorithm. 


V TOPOLOGICAL ALGORITHMS 


17 Geometric Graphs 
18 Surfaces 
19 Homology 
Fifth Homework Assignment 


17 Geometric Graphs 


In the abstract notion of a graph, an edge is merely a pair of 
vertices. The geometric (or topological) notion of a graph 
is closer to our intuition in which we think of an edge as a 
curve that connects two vertices. 


Embeddings. Let G = (V, E) be a simple, undirected 
graph and write R? for the two-dimensional real plane. 
A drawing maps every vertex v € V to a point e(v) in 
R?, and it maps every edge {u,v} € E to a curve with 
endpoints e(u) and ¢(v). The drawing is an embedding if 


1. different vertices map to different points; 
2. the curves have no self-intersections; 


3. the only points of a curve that are images of vertices 
are its endpoints; 


4. two curves intersect at most in their endpoints. 


We can always map the vertices to points and the edges 
to curves in R? so they form an embedding. On the other 
hand, not every graph has an embedding in R?. The graph 
G is planar if it has an embedding in R?. As illustrated 
in Figure 73, a planar graph has many drawings, not all of 
which are embeddings. A straight-line drawing or embed- 


Figure 73: Three drawings of K4, the complete graph with four 
vertices. From left to right: a drawing that is not an embedding, 
an embedding with one curved edge, a straight-line embedding. 


ding is one in which each edge is mapped to a straight line 
segment. It is uniquely determined by the mapping of the 
vertices, e : V — R?. We will see later that every planar 
graph has a straight-line embedding. 


Euler’s formula. A face of an embedding £ of G is a 
component of the thus defined decomposition of R?. We 
write n = |V|, m = |E], and £ for the number of faces. 
Euler’s formula says these numbers satisfy a linear rela- 
tion. 


EULER’S FORMULA. If G is connected and £ is an em- 
bedding of G in R? then n — m+ l = 2. 
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PROOF. Choose a spanning tree (V, T) of G = (V, E). It 
has n vertices, |T| = n — 1 edges, and one (unbounded) 
face. We have n — (n — 1) + 1 = 2, which proves the for- 
mula if G is a tree. Otherwise, draw the remaining edges, 
one at a time. Each edge decomposes one face into two. 
The number of vertices does not change, m increases by 
one, and £ increases by one. Since the graph satisfies the 
linear relation before drawing the edge, it satisfies the re- 
lation also after drawing the edge. 


A planar graph is maximally connected if adding any 
one new edge violates planarity. Not surprisingly, a planar 
graph of three or more vertices is maximally connected 
iff every face in an embedding is bounded by three edges. 
Indeed, suppose there is a face bounded by four or more 
edges. Then we can find two vertices in its boundary that 
are not yet connected and we can connect them by draw- 
ing a curve that passes through the face; see Figure 74. 
For obvious reasons, we call an embedding of a maxi- 


Figure 74: Drawing the edge from a to c decomposes the quad- 
rangle into two triangles. Note that we cannot draw the edge 
from b to d since it already exists outside the quadrangle. 


mally connected planar graph with n > 3 vertices a tri- 
angulation. For such graphs, we have an additional linear 
relation, namely 3€ = 2m. We can thus rewrite Euler’s 
formula and get n — m + 22 = 2andn— Yil= 2 and 
therefore 


3n — 6; 
2n — 4, 


m = 


4 = 


Every planar graph can be completed to a maximally con- 
nected planar graph. For n > 3 this implies that the planar 
graph has at most 3n — 6 edges and at most 2n — 4 faces. 


Forbidden subgraphs. We can use Euler’s relation to 
prove that the complete graph of five vertices is not planar. 
It has n = 5 vertices and m = 10 edges, contradicting the 
upper bound of at most 3n — 6 = 9 edges. Indeed, every 
drawing of Ks has at least two edges crossing; see Figure 
75. Similarly, we can prove that the complete bipartite 


Figure 75: A drawing of Ks on the left and of K3,3 on the right. 


graph with three plus three vertices is not planar. It has 
n = 6 vertices and m = 9 edges. Every cycle in a bipartite 
graph has an even number of edges. Hence, 40 < 2m. 
Plugging this into Euler’s formula, we get n- m+ 4 > 2 
and therefore m < 2n — 4 = 8, again a contradiction. 


In a sense, K5 and K3, 3 are the quintessential non- 
planar graphs. To make this concrete, we still need an 
operation that creates or removes degree-2 vertices. Two 
graphs are homeomorphic if one can be obtained from the 
other by a sequence of operations, each deleting a degree-2 
vertex and replacing its two edges by the one that connects 
its two neighbors, or the other way round. 


KURATOWSKI’S THEOREM. A graph G is planar iff no 
subgraph of G is homeomorphic to Ks or to K3.3. 


The proof of this result is a bit lengthy and omitted. 


Pentagons are star-convex. Euler’s formula can also be 
used to show that every planar graph has a straight-line 
embedding. Note that the sum of vertex degrees counts 
each edge twice, that is, $` „ey deg(v) = 2m. For planar 
graphs, twice the number of edges is less than 6n which 
implies that the average degree is less than six. It follows 
that every planar graph has at least one vertex of degree 
5 or less. This can be strengthened by saying that every 
planar graph with n > 4 vertices has at least four vertices 
of degree at most 5 each. To see this, assume the planar 
graph is maximally connected and note that every vertex 
has degree at least 3. The deficiency from degree 6 is thus 
at most 3. The total deficiency is 6n — X` „ṣ¿y deg(v) = 
12 which implies that we have at least four vertices with 
positive deficiency. 


We need a little bit of geometry to prepare the construc- 
tion of a straight-line embedding. A region R C R? is 
convex if x,y € R implies that the entire line segment 
connecting x and y is contained in R. Figure 76 shows 
regions of either kind. We call R star-convex of there is 
a point z € R such that for every point x € R the line 
segment connecting x with z is contained in R. The set of 
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Figure 76: A convex region on the left and a non-convex star- 
convex region on the right. 


such points z is the kernel of R. Clearly, every convex re- 
gion is star-convex but not every star-convex region is con- 
vex. Similarly, there are regions that are not star-convex, 
even rather simple ones such as the hexagon in Figure 77. 
However, every pentagon is star-convex. Indeed, the pen- 


Figure 77: A non-star-convex hexagon on the left and a star- 
convex pentagon on the right. The dark region inside the pen- 
tagon is its kernel. 


tagon can be decomposed into three triangles by drawing 
two diagonals that share an endpoint. Extending the inci- 
dent sides into the pentagon gives locally the boundary of 
the kernel. It follows that the kernel is non-empty and has 
interior points. 


Fary’s construction. We construct a straight-line em- 
bedding of a planar graph G = (V, E) assuming G is 
maximally connected. Choose three vertices, a, b, c, con- 
nected by three edges to form the outer triangle. If G has 
only n = 3 vertices we are done. Else it has at least one 
vertex u € V = {a,b,c} with deg(u) < 5. 


Step 1. Remove u together with the k = deg(u) edges 
incident to u. Add k — 3 edges to make the graph 
maximally connected again. 


Step 2. Recursively construct a straight-line embed- 
ding of the smaller graph. 


Step 3. Remove the added k — 3 edges and map u to 
a point e(u) in the interior of the kernel of the result- 
ing k-gon. Connect e(u) with line segments to the 
vertices of the k-gon. 


Figure 78 illustrates the recursive construction. It is 
straightforward to implement but there are numerical is- 
sues in the choice of ¢(u) that limit the usefulness of this 


construction. 
| 
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| removeu | 
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Figure 78: We fix the outer triangle, remove the degree-5 vertex, 
recursively construct a straight-line embedding of the rest, and 
finally add the vertex back. 


Tutte’s construction. A more useful construction of a 
straight-line embedding goes back to the work of Tutte. 
We begin with a definition. Given a finite set of points, 
£1, T2, .. . , Zj, the average is 


J 
1 
Tus 
w=1 


For j = 2, it is the midpoint of the edge and for 7 = 3, 
it is the centroid of the triangle. In general, the average 
is a point somewhere between the x;. Let G = (V, E) 
be a maximally connected planar graph and a, b,c three 
vertices connected by three edges. We now follow Tutte’s 
construction to get a mapping € : V — R? so that the 
straight-line drawing of G is a straight-line embedding. 


HA = 


Step 1. Map a,b,c to points e(a), e(b), e(c) spanning 
a triangle in R?. 


Step 2. For each vertex u € V — {a,b,c}, let N, be 
the set of neighbors of u. Map u to the average of the 
images of its neighbors, that is, 


elu) = mm 5 e(v). 


vENu 
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The fact that the resulting mapping £ : V — R? gives a 
straight-line embedding of G is known as Tutte’s Theo- 
rem. It holds even if G is not quite maximally connected 
and if the points are not quite the averages of their neigh- 
bors. The proof is a bit involved and omitted. 


The points ¢(w) can be computed by solving a system of 
linear equations. We illustrate this for the graph in Figure 
78. We set e(a) = (Z1), e(b) = (1), elc) = (9). The 
other five points are computed by solving the system of 


linear equations Av = 0, where 


II 
ooro°9o 
Orr OoO 
a 
eee oO 


and v is the column vector of points e(a) to e(y). There 
are really two linear systems, one for the horizontal and 
the other for the vertical coordinates. In each system, we 
have n — 3 equations and a total of n — 3 unknowns. This 
gives a unique solution provided the equations are linearly 
independent. Proving that they are is part of the proof of 
Tutte’s Theorem. Solving the linear equations is a numeri- 
cal problem that is studies in detail in courses on numerical 
analysis. 


18 Surfaces 


Graphs may be drawn in two, three, or higher dimen- 
sions, but they are still intrinsically only 1-dimensional. 
One step up in dimensions, we find surfaces, which are 
2-dimensional. 


Topological 2-manifolds. The simplest kind of surfaces 
are the ones that on a small scale look like the real plane. 
A space M is a 2-manifold if every point x € M is 
locally homeomorphic to R?. Specifically, there is an 
open neighborhood N of x and a continuous bijection 
h : N — R? whose inverse is also continuous. Such a 
bicontinuous map is called a homeomorphism. Examples 
of 2-manifolds are the open disk and the sphere. The for- 
mer is not compact because it has covers that do not have 
finite subcovers. Figure 79 shows examples of compact 2- 
manifolds. If we add the boundary circle to the open disk 
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Figure 79: Three compact 2-manifolds, the sphere, the torus, and 
the double torus. 


we get a closed disk which is compact but not every point 
is locally homeomorphic to R?. Specifically, a point on 
the circle has an open neighborhood homeomorphic to the 
closed half-plane, H? = {(x1, £2) E€ R? | zı > 0}. A 
space whose points have open neighborhoods homeomor- 
phic to R? or H? is called a 2-manifolds with boundary; 
see Figure 80 for examples. The boundary is the subset 
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Figure 80: Three 2-manifolds with boundary, the closed disk, the 
cylinder, and the Mobius strip. 


of points with neighborhoods homeomorphic to H?. It is 
a 1-manifold (without boundary), that is, every point is 
locally homeomorphic to R. There is only one type of 
compact, connected 1-manifold, namely the closed curve. 
In topology, we do not distinguish spaces that are home- 
omorphic to each other. Hence, every closed curve is like 
every other one and they are all homeomorphic to the unit 
circle, S! = {x € R? | |||] = 1}. 
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Triangulations. A standard representation of a compact 
2-manifold uses triangles that are glued to each other 
along shared edges and vertices. A collection K of tri- 
angles, edges, and vertices is a triangulation of a compact 
2-manifold if 


I. for every triangle in K, its three edges belong to K, 
and for every edge in K, its two endpoints are ver- 
tices in K; 


II. every edge belongs to exactly two triangles and every 
vertex belongs to a single ring of triangles. 


An example is shown in Figure 81. To simplify language, 
we call each element of K a simplex. If we need to be spe- 
cific, we add the dimension, calling a vertex a 0-simplex, 
an edge a 1-simplex, and a triangle a 2-simplex. A face 
of a simplex 7 is a simplex ø C 7. For example, a trian- 
gle has seven faces, its three vertices, its two edges, and 
itself. We can now state Condition I more succinctly: if 
a is a face of r and T E€ K then o € K. To talk about 
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Figure 81: A triangulation of the sphere. The eight triangles are 
glued to form the boundary of an octahedron which is homeo- 
morphic to the sphere. 


the inverse of the face relation, we define the star of a 
simplex o as the set of simplices that contain o as a face, 
Sto = {r E€ K | o C T}. Sometimes we think of the 
star as a set of simplices and sometimes as a set of points, 
namely the union of interiors of the simplices in the star. 
With the latter interpretation, we can now express Condi- 
tion II more succinctly: the star of every simplex in K is 
homeomorphic to R?. 


Data structure. When we store a 2-manifold, it is use- 
ful to keep track of which side we are facing and where 
we are going so that we can move around efficiently. 
The core piece of our data structure is a representation 
of the symmetry group of a triangle. This group is iso- 
morphic to the group of permutations of three elements, 


the vertices of the triangle. We call each permutation 
an ordered triangle and use cyclic shifts and transposi- 
tions to move between them; see Figure 82. We store 
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Figure 82: The symmetry group of the triangle consists of six 
ordered versions. Each ordered triangle has a lead vertex and a 
lead directed edge. 


the entire symmetry group in a single node of an abstract 
graph, with arcs between neighboring triangles. Further- 
more, we store the vertices in a linear array, V[1..n]. For 
each ordered triangle, we store the index of the lead ver- 
tex and a pointer to the neighboring triangle that shares 
the same directed lead edge. A pointer in this context 
is the address of a node together with a three-bit inte- 
ger, 4, that identifies the ordered version of the triangle 
we refer to. Suppose for example that we identify the 
ordered versions abc, bca, cab, bac, cba, acb of a triangle 
with ¿ = 0,1,2,4,5,6, in this sequence. Then we can 
move between different ordered versions of the same tri- 
angle using the following functions. 


ordTri ENEXT(u, t) 
ifv<2then return (p, (c 


Lt, (t + 1) mod 3) 
else return (u, (t + 


1) mod 3 + 4) 
endif. 


ordTri SYM(u, +) 
return (u, (t + 4) mod 8). 


To get the index of the lead vertex, we use the integer func- 
tion ORG(u,.) and to get the pointer to the neighboring 
triangle, we use FNEXT(p, /). 


Orientability. A 2-manifold is orientable if it has two 
distinct sides, that is, if we move around on one we stay 
there and never cross over to the other side. The one exam- 
ple of a non-orientable manifold we have seen so far is the 
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MObious strip in Figure 80. There are also non-orientable, 
compact 2-manifolds (without boundary), as we can see in 
Figure 83. We use the data structure to decide whether or 


E 


Figure 83: Two non-orientable, compact 2-manifolds, the pro- 
jective plane on the left and the Klein bottle on the right. 


not a 2-manifold is orientable. Note that the cyclic shift 
partitions the set of six ordered triangles into two orien- 
tations, each consisting of three triangles. We say two 
neighboring triangles are consistently oriented if they dis- 
agree on the direction of the shared edge, as in Figure 81. 
Using depth-first search, we visit all triangles and orient 
them consistently, if possible. At the first visit, we ori- 
ent the triangle consistent with the preceding, neighboring 
triangle. At subsequence visits, we check for consistent 
orientation. 


boolean ISORNTBL(J, t) 

if wis unmarked then 
mark u; choose the orientation that contains 1; 
bx = ISORNTBL(FNEXT(SYM(ts, 1))); 
by = ISORNTBL(FNEXT(ENEXT(SYM({1, 4)))); 
bz = ISORNTBL(FNEXT(ENEXT?(SYM(jJ, L)))); 
return br and by and b; 

else 
return [orientation of u contains 1] 

endif. 


There are two places where we return a boolean value. At 
the second place, it indicates whether or not we have con- 
sistent orientation in spite of the visited triangle being ori- 
ented prior to the visit. At the first place, the boolean value 
indicates whether or not we have found a contradiction to 
orientablity so far. A value of FALSE anywhere during the 
computation is propagated to the root of the search tree 
telling us that the 2-manifold is non-orientable. The run- 
ning time is proportional to the number of triangles in the 
triangulation of the 2-manifold. 


Classification. For the sphere and the torus, it is easy 
to see how to make them out of a sheet of paper. Twist- 
ing the paper gives a non-orientable 2-manifold. Perhaps 


most difficult to understand is the projective plane. It is 
obtained by gluing each point of the sphere to its antipodal 
point. This way, the entire northern hemisphere is glued 
to the southern hemisphere. This gives the disk except 
that we still need to glue points of the bounding circle (the 
equator) in pairs, as shown in the third paper construction 
in Figure 84. The Klein bottle is easier to imagine as it 
is obtained by twisting the paper just once, same as in the 
construction of the Mobius strip. 


a a a a 


Figure 84: From left to right: the sphere, the torus, the projective 
plane, and the Klein bottle. 


There is a general method here that can be used to clas- 
sify the compact 2-manifolds. Given two of them, we con- 
struct a new one by removing an open disk each and glu- 
ing the 2-manifolds along the two circles. The result is 
called the connected sum of the two 2-manifolds, denoted 
as MN. For example, the double torus is the connected 
sum of two tori, T?#4T?. We can cut up the g-fold torus 
into a flat sheet of paper, and the canonical way of doing 
this gives a 4g-gon with edges identified in pairs as shown 
in Figure 85 on the left. The number g is called the genus 
of the manifold. Similarly, we can get new non-orientable 
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Figure 85: The polygonal schema in standard form for the double 
torus and the double Klein bottle. 


manifolds from the projective plane, P?, by forming con- 
nected sums. Cutting up the g-fold projective plane gives 
a 2g-gon with edges identified in pairs as shown in Figure 
85 on the right. We note that the constructions of the pro- 
jective plane and the Klein bottle in Figure 84 are both not 
in standard form. A remarkable result which is now more 
than a century old is that every compact 2-manifold can be 
cut up to give a standard polygonal schema. This implies 
a classification of the possibilities. 
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CLASSIFICATION THEOREM. The members of the fami- 
lies S?, T?, T?#T?,... and P?, P?#P?,... are non- 
homeomorphic and they exhaust the family of com- 
pact 2-manifolds. 


Euler characteristic. Suppose we are given a triangula- 
tion, K, of a compact 2-manifold, M. We already know 
how to decide whether or not M is orientable. To deter- 
mine its type, we just need to find its genus, which we do 
by counting simplices. The Euler characteristic is 

x = #vertices — #edges + #triangles. 
Let us look at the orientable case first. We have a 4g-gon 
which we triangulate. This is a planar graph with n — 
m + £ = 2. However, 2g edge are counted double, the 4g 
vertices of the 4g-gon are all the same, and the outer face 
is not a triangle in K. Hence, 


x = (n-4g+1) 
(n—m-+ £) — 2g 


2g) + (£— 1) 


(m 


which is equal to 2 — 2g. The same analysis can be used 
in the non-orientable case in which we get x = (n — 2g + 
1) — (m — g) + (£ — 1) = 2 — g. To decide whether 
two compact 2-manifolds are homeomorphic it suffices to 
determine whether they are both orientable or both non- 
orientable and, if they are, whether they have the same 
Euler characteristic. This can be done in time linear in the 
number of simplices in their triangulations. 


This result is in sharp contrast to the higher-dimensional 
case. The classification of compact 3-manifolds has been 
a longstanding open problem in Mathematics. Perhaps 
the recent proof of the Poincaré conjecture by Perelman 
brings us close to a resolution. Beyond three dimensions, 
the situation is hopeless, that is, deciding whether or not 
two triangulated compact manifolds of dimension four or 
higher are homeomorphic is undecidable. 


19 Homology 


In topology, the main focus is not on geometric size but 
rather on how a space is connected. The most elementary 
notion distinguishes whether we can go from one place 
to another. If not then there is a gap we cannot bridge. 
Next we would ask whether there is a loop going around 
an obstacle, or whether there is a void missing in the space. 
Homology is a formalization of these ideas. It gives a way 
to define and count holes using algebra. 


The cyclomatic number of a graph. To motivate the 
more general concepts, consider a connected graph, G, 
with n vertices and m edges. A spanning tree has n — 1 
edges and every additional edge forms a unique cycle to- 
gether with edges in this tree; see Figure 86. Every other 


Figure 86: A tree with three additional edges defining the same 
number of cycles. 


cycle in G can be written as a sum of these m — (n — 1) 
cycles. To make this concrete, we define a cycle as a sub- 
set of the edges such that every vertex belongs to an even 
number of these edges. A cycle does not need to be con- 
nected. The sum of two cycles is the symmetric difference 
of the two sets such that multiple edges erase each other 
in pairs. Clearly, the sum of two cycles is again a cy- 
cle. Every cycle, y, in G contains some positive number 
of edges that do not belong to the spanning tree. Call- 
ing these edges e1, €2,..., €x and the cycles they define 
Y1,2,+++5 Yk, we claim that 
Y = HERE... + Vk 

To see this assume that ô = y1 + Y2 + . - - + Yk is different 
from y. Then y+ô is again a cycle but it contains no edges 
that do not belong to the spanning tree. Hence y + 6 = 0) 
and therefore y = ô, as claimed. This implies that the 
m-— n+ 1 cycles form a basis of the group of cycles which 
motivates us to call m — n + 1 the cyclomatic number of 
the graph. Note that the basis depends on the choice of 
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spanning tree while the cyclomatic number is independent 
of that choice. 


Simplicial complexes. We begin with a combinatorial 
representation of a topological space. Using a finite 
ground set of vertices, V, we call a subset o C V an 
abstract simplex. Its dimension is one less than the car- 
dinality, dim ø = |o| — 1. A face is a subset T C ø. 


DEFINITION. An abstract simplicial complex over V is a 
system K C 2” such that o € K and 7 C ø implies 
TEK. 


The dimension of K is the largest dimension of any sim- 
plex in K. A graph is thus a 1-dimensional abstract sim- 
plicial complex. Just like for graphs, we sometimes think 
of K as an abstract structure and at other times as a geo- 
metric object consisting of geometric simplices. In the lat- 
ter interpretation, we glue the simplices along shared faces 
to form a geometric realization of K, denoted as | K|. We 
say K triangulates a space X if there is a homeomorphism 
h : X — |K|. We have seen 1- and 2-dimensional exam- 
ples in the preceding sections. The boundary of a simplex 
o is the collection of co-dimension one faces, 
ðo = {rC |dimr = dimo — 1}. 

If dimo = p then the boundary consists of p + 1 (p — 1)- 
simplices. Every (p — 1)-simplex has p (p — 2)-simplices 
in its own boundary. This way we get (p + 1)p (p — 2)- 
simplices, counting each of the aa = ("3") (p — 2)- 
dimensional faces of o twice. 


Chain complexes. We now generalize the cycles in 
graphs to cycles of different dimensions in simplicial com- 
plexes. A p-chain is a set of p-simplices in K. The sum 
of two p-chains is their symmetric difference. We usually 
write the sets as formal sums, 


Cc = 


d = 


a101 +a202 +... + nOn; 


bici + b202 Fus + bron, 


where the a; and b; are either 0 or 1. Addition can then be 
done using modulo 2 arithmetic, 


ctod = (a, +2 b1)o1 +...+ (an +2 On)on, 


where a; +2 b; is the exclusive or operation. We simplify 
notation by dropping the subscript but note that the two 
plus signs are different, one modulo two and the other a 
formal notation separating elements in a set. The p-chains 


form a group, which we denote as (Cp, +) or simply Cp. 
Note that the boundary of a p-simplex is a (p — 1)-chain, 
an element of C,_1. Extending this concept linearly, we 
define the boundary of a p-chain as the sum of boundaries 
of its simplices, Oc = a,00,+...+4,00,. The boundary 
is thus a map between chain groups and we sometimes 
write the dimension as index for clarity, 


Ops Ce Cp aie 


It is a homomorphism since 0,(¢ + d) = Opc + Opd. The 
infinite sequence of chain groups connected by boundary 
homomorphisms is called the chain complex of K. All 
groups of dimension smaller than 0 and larger than the di- 
mension of K are trivial. It is convenient to keep them 
around to avoid special cases at the ends. A p-cycle is a 
p-chain whose boundary is zero. The sum of two p-cycles 
is again a p-cycle so we get a subgroup, Zp C Cp. A 
p-boundary is a p-chain that is the boundary of a (p + 1)- 
chain. The sum of two p-boundaries is again a p-boundary 
so we get another subgroup, B, C C,, Taking the bound- 
ary twice in a row gives zero for every simplex and thus 
for every chain, that is, (0,(0,41d) = 0. It follows that 
B, is a subgroup of Zp. We can therefore draw the chain 
complex as in Figure 87. 


Figure 87: The chain complex consisting of a linear sequence 
of chain, cycle, and boundary groups connected by homomor- 
phisms. 


Homology groups. We would like to talk about cycles 
but ignore the boundaries since they do not go around a 
hole. At the same time, we would like to consider two 
cycles the same if they differ by a boundary. See Figure 
88 for a few 1-cycles, some of which are 1-boundaries and 
some of which are not. This is achieved by taking the 
quotient of the cycle group and the boundary group. The 
result is the p-th homology group, 


Hp =. Zp/Bp: 

Its elements are of the form [c] = c + Bp, where c is a p- 
cycle. [c] is called a homology class, c is a representative 
of [c], and any two cycles in [c] are homologous denoted 
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Figure 88: The 1-cycles y and ô are not 1-boundaries. Adding 
the 1-boundary € to ô gives a 1-cycle homologous to 6. 


as c ~ c’. Note that [c] = [c’] whenever c ~ c’. Also note 
that [c + d] = [t + d'] whenever c ~ c’ and d ~ d’. We 
use this as a definition of addition for homology classes, so 
we again have a group. For example, the 1-st homology 
group of the torus consists of four elements, [0] = Bi, 
ly] = y+ B1, [6] = ô + B1, and [y+ 6] = y+ ô+ B1. We 
often draw the elements as the corners of a cube of some 
dimension; see Figure 89. If the dimension is 8 then it has 
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Figure 89: The four homology classes of Hı are generated by 
two classes, [y] and [ô]. 


2° corners. The dimension is also the number of classes 
needed to generate the group, the size of the basis. For 
the p-th homology group, this number is 6, = rank Hp = 
logy |H,|, the p-th Betti number. For the torus we have 


Bo = l; 
By = 2; 
By = L; 


and p = 0 for all p # 0,1,2. Every 0-chain is a 0- 
cycle. Two 0-cycles are homologous if they are both the 
sum of an even number or both of an odd number of ver- 
tices. Hence G9 = loga 2 = 1. We have seen the reason 
for 3; = 2 before. Finally, there are only two 2-cycles, 
namely 0 and the set of all triangles. The latter is not a 
boundary, hence 32 = log, 2 = 1. 


Boundary matrices. To compute homology groups and 
Betti numbers, we use a matrix representation of the sim- 
plicial complex. Specifically, we store the boundary ho- 
momorphism for each dimension, setting 0,[i, j] = 1 if 


the i-th (p — 1)-simplex is in the boundary of the j-th p- 
simplex, and O,[i, j] = 0, otherwise. For example, if the 
complex consists of all faces of the tetrahedron, then the 
boundary matrices are 


i = [0 0 0 0]; 
11100 0 
i 100110], 
“F 010101? 
001011 
1 100 
1 010 
0110 
ô = LO. 2. 
0101 
0 0 1 1 
1 
1 
ð = |] 
1 


Given a p-chain as a column vector, v, its boundary is 
computed by matrix multiplication, 0,v. The result is a 
combination of columns in the p-th boundary matrix, as 
specified by v. Thus, v is a p-cycle iff 0,v = 0 and v is a 
p-boundary iff there is u such that 0,,;u = v. 


Matrix reduction. Letting np be the number of p- 
simplices in K, we note that it is also the rank of the p-th 
chain group, np = rank Cp. The p-th boundary matrix 
thus has np—ı rows and np columns. To figure the sizes of 
the cycle and boundary groups, and thus of the homology 
groups, we reduce the matrix to normal form, as shown 
in Figure 90. The algorithm of choice uses column and 


—— rank C p Se 


É- rank Zp = 


rankC,_1 | 


Figure 90: The p-th boundary matrix in normal form. The entries 
in the shaded portion of the diagonal are 1 and all other entries 
are 0. 


row operations similar to Gaussian elimination for solv- 


ing a linear system. We write it recursively, calling it with 
m= 


void REDUCE(m) 
if dk,l > m with [k,l] = 1 then 
exchange rows m and k and columns m and l; 
fori =m +1 to np- do 
if pli, m] = 1 then 
add row m to row i 
endif 
endfor; 
for jJ =m + 1to np do 
if 0,[m,j] = 1 then 
add column m to column j 
endif 
endfor; 
REDUCE(m + 1) 
endif. 


For each recursive call, we have at most a linear number 
of row and column operations. The total running time is 
therefore at most cubic in the number of simplices. Figure 
90 shows how we interpret the result. Specifically, the 
number of zero columns is the rank of the cycle group, 
Zp, and the number of 1s in the diagonal is the rank of the 
boundary group, Bp—1. The Betti number is the difference, 


Bp 


taking the rank of the boundary group from the reduced 
matrix one dimension up. Working on our example, we 
get the following reduced matrices. 


rank Z, — rank By, 


= [0 00 0]; 
100000 
a 010 00 0 
/_ 0010004" 
000 0 0 0 
100 0 
010 0 
001 0 
ô = 000 0]: 
000 0 
000 0 
1 
0 
ð = 3 
0 


Writing z, = rank Zņ and bp = rank By, we get z = 4 
from the zeroth and bo = 3 from the first reduced bound- 
ary matrix. Hence 6o zo = bo = 1. Furthermore, 
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zı = 3and bı = 3 giving 6; = 0, z2 = 1l and b2 = 1 
giving 32 = 0, and z3 = 0 giving 33 = 0. These are the 
Betti numbers of the closed ball. 


Euler-Poincaré Theorem. The Euler characteristic of a 
simplicial complex is the alternating sum of simplex num- 
bers, 


x = S75 (-1)? np. 


p20 


Recalling that n, is the rank of the p-th chain group and 
that it equals the rank of the p-th cycle group plus the rank 
of the (p — 1)-st boundary group, we get 


x = DOH)? (ep + bp) 


which is the same as the alternating sum of Betti num- 
bers. To appreciate the beauty of this result, we need to 
know that the Betti numbers do not depend on the trian- 
gulation chosen for the space. The proof of this property 
is technical and omitted. This now implies that the Euler 
characteristic is an invariant of the space, same as the Betti 
numbers. 


EULER-POINCARE THEOREM. x =} (—1)’ 8p. 
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Fifth Homework Assignment 


Write the solution to each problem on a single page. The 
deadline for handing in solutions is November 13. 


Problem 1. (20 points). Let G = (V,£E) be a maxi- 
mally connected planar graph and recall that [k] = 
{1,2,...,k}. A vertex k-coloring is a mapping y : 
V — [k] such that y(u) 4 y(v) whenever u # v 
are adjacent, and an edge k-coloring is a mapping 
n : E — [|k] such that n(e) 4 n( f) whenever e 4 f 
bound a common triangle. Prove that if G has a ver- 
tex 4-coloring then it also has an edge 3-coloring. 


Problem 2. (20 = 10 + 10 points). Let K be a set of 
triangles together with their edges and vertices. The 
vertices are represented by a linear array, as usual, but 
there is no particular ordering information in the way 
the edges and triangles are given. In other words, the 
edges are just a list of index pairs and the triangles 
are a list of index triplets into the vertex array. 


(a) Give an algorithm that decides whether or not 
K is a triangulation of a 2-manifold. 

(b) Analyze your algorithm and collect credit 
points if the running time of your algorithm is 
linear in the number of triangles. 


Problem 3. (20 = 5+7+8 points). Determine the type of 
2-manifold with boundary obtained by the following 
constructions. 


(a) Remove a cylinder from a torus in such a way 
that the rest of the torus remains connected. 


(b) Remove a disk from the projective plane. 
(c) Remove a Möbius strip from a Klein bottle. 


Whenever we remove a piece, we do this like cutting 
with scissors so that the remainder is still closed, in 
each case a 2-manifold with boundary. 


Problem 4. (20 = 5 + 5 + 5 + 5 points). Recall that the 
sphere is the space of points at unit distance from the 
origin in three-dimensional Euclidean space, S? = 
{x € R? | ||x|| = 1}. 


(a) Give a triangulation of S?. 

(b) Give the corresponding boundary matrices. 
(c) Reduce the boundary matrices. 

(d) Give the Betti numbers of S?. 
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Problem 5. (20 = 10 + 10 points). The dunce cap is ob- 


tained by gluing the three edges of a triangular sheet 
of paper to each other. [After gluing the first two 
edges you get a cone, with the glued edges forming a 
seam connecting the cone point with the rim. In the 
final step, wrap the seam around the rim, gluing all 
three edges to each other. To imagine how this work, 
it might help to think of the final result as similar to 
the shell of a snale. | 


(a) Is the dunce cap a 2-manifold? Justify your an- 
Swer. 

(b) Give a triangulation of the dunce cap, making 
sure that no two edges connect the same two 
vertices and no two triangles connect the same 
three vertices. 
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20 Plane-Sweep 


Plane-sweep is an algorithmic paradigm that emerges in 
the study of two-dimensional geometric problems. The 
idea is to sweep the plane with a line and perform the com- 
putations in the sequence the data is encountered. In this 
section, we solve three problems with this paradigm: we 
construct the convex hull of a set of points, we triangulate 
the convex hull using the points as vertices, and we test a 
set of line segments for crossings. 


Convex hull. Let S be a finite set of points in the plane, 
each given by its two coordinates. The convex hull of S, 
denoted by conv S, is the smallest convex set that con- 
tains S. Figure 91 illustrates the definition for a set of 
nine points. Imagine the points as solid nails in a planar 
board. An intuitive construction stretches a rubber band 
around the nails. After letting go, the nails prevent the 
complete relaxation of the rubber band which will then 
trace the boundary of the convex hull. 


50 
TO 


3 6 


Figure 91: The convex hull of nine points, which we represent 
by the counterclockwise sequence of boundary vertices: 1, 3, 6, 
8, 9, 2: 


To construct the counterclockwise cyclic sequence of 
boundary vertices representing the convex hull, we sweep 
a vertical line from left to right over the data. At any mo- 
ment in time, the points in front (to the right) of the line 
are untouched and the points behind (to the left) of the line 
have already been processed. 


Step 1. Sort the points from left to right and relabel 
them in this sequence as £1, %2,..., En. 


Step 2. Construct a counterclockwise triangle from 
the first three points: 71 72%3 OF %1%3%2. 


Step 3. For i from 4 to n, add the next point x; to the 
convex hull of the preceding points by finding the 
two lines that pass through x; and support the con- 
vex hull. 
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The algorithm is illustrated in Figure 92, which shows the 
addition of the sixth point in the data set. 


Figure 92: The vertical sweep-line passes through point 6. To 
add 6, we substitute 6 for the sequence of vertices on the bound- 
ary between 3 and 5. 


Orientation test. A critical test needed to construct the 
convex hull is to determine the orientation of a sequence 
of three points. In other words, we need to be able to dis- 
tinguish whether we make a left-turn or a right-turn as we 
go from the first to the middle and then the last point in 
the sequence. A convenient way to determine the orien- 
tation evaluates the determinant of a three-by-three ma- 
trix. More precisely, the points a = (a1, a2), b = (b1, b2), 
c = (c1, C2) form a left-turn iff 


1 a, Q2 
det 1 by bə > 0. 
1 C1 C2 


The three points form a right-turn iff the determinant is 
negative and they lie on a common line iff the determinant 
is zero. 


boolean LEFT(Points a, b,c) 
return [a1(b2 — ce) + bı (c2 — a2) 
+ cı (az — bz) > O]. 


To see that this formula is correct, we may convince our- 
selves that it is correct for three non-collinear points, e.g. 
a = (0,0), b = (1,0), and c = (0,1). Remember also 
that the determinant measures the area of the triangle and 
is therefore a continuous function that passes through zero 
only when the three points are collinear. Since we can 
continuously move every left-turn to every other left-turn 
without leaving the class of left-turns, it follows that the 
sign of the determinant is the same for all of them. 


Finding support lines. We use a doubly-linked cyclic 
list of vertices to represent the convex hull boundary. Each 


node in the list contains pointers to the next and the previ- 
ous nodes. In addition, we have a pointer last to the last 
vertex added to the list. This vertex is also the rightmost 
in the list. We add the 2-th point by connecting it to the 
vertices u — pt and À — pt identified in a counterclock- 
wise and a clockwise traversal of the cycle starting at last, 
as illustrated in Figure 93. We simplify notation by using 


u 
Q ae ee 
O last 6 v 
d ee 

À 


Figure 93: The upper support line passes through the first point 
H — pt that forms a left-turn from v —> pt to u —> next — pt. 


nodes in the parameter list of the orientation test instead 
of the points they store. 


u = à = last; create new node with v — pt = i; 
while RIGHT(y, u, u — next) do 
b= H — next 
endwhile; 
while LEFT(v, A, \ — prev) do 
A= X= prev 
endwhile; 
v — nett = u, v > prev = À; 
u > prev = à > next =v; last = v. 


The effort to add the ż-th point can be large, but if it is 
then we remove many previously added vertices from the 
list. Indeed, each iteration of the for-loop adds only one 
vertex to the cyclic list. We charge $2 for the addition, 
one dollar for the cost of adding and the other to pay for 
the future deletion, if any. The extra dollars pay for all 
iterations of the while-loops, except for the first and the 
last. This implies that we spend only constant amortized 
time per point. After sorting the points from left to right, 
we can therefore construct the convex hull of n points in 
time O(n). 


Triangulation. The same plane-sweep algorithm can be 
used to decompose the convex hull into triangles. All 
we need to change is that points and edges are never re- 
moved and a new point is connected to every point exam- 
ined during the two while-loops. We define a (geometric) 
triangulation of a finite set of points S in the plane as a 
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maximally connected straight-line embedding of a planar 
graph whose vertices are mapped to points in S. Figure 94 
shows the triangulation of the nine points in Figure 91 con- 
structed by the plane-sweep algorithm. A triangulation is 


Figure 94: Triangulation constructed with the plane-sweep algo- 
rithm. 


not necessarily a maximally connected planar graph since 
the prescribed placement of the points fixes the boundary 
of the outer face to be the boundary of the convex hull. 
Letting k be the number of edges of that boundary, we 
would have to add k — 3 more edges to get a maximally 
connected planar graph. It follows that the triangulation 
has m = 3n — (k + 3) edges and £ = 2n — (k + 2) 
triangles. 


Line segment intersection. As a third application of the 
plane-sweep paradigm, we consider the problem of decid- 
ing whether or not n given line segments have pairwise 
disjoint interiors. We allow line segments to share end- 
points but we do not allow them to cross or to overlap. We 
may interpret this problem as deciding whether or not a 
straight-line drawing of a graph is an embedding. To sim- 
plify the description of the algorithm, we assume no three 
endpoints are collinear, so we only have to worry about 
crossings and not about other overlaps. 


How can we decide whether or not a line segment 
with endpoint u = (u1, u2) and v = (v1, v2) crosses 
another line segment with endpoints p = (pı, p2) and 
q = (q1, 92)? Figure 95 illustrates the question by show- 
ing the four different cases of how two line segments and 
the lines they span can intersect. The line segments cross 
iff uv intersects the line of pq and pq intersects the line of 
uv. This condition can be checked using the orientation 
test. 


boolean CROSS(Points u,v, p,q) 
return [(LEFT(u, v, p) xor LEFT(u, v, q)) and 
(LEFT(p, q, u) xor LEFT(p, q, v))]. 


We can use the above function to test all (5) pairs of line 
segments, which takes time O(n?). 


LY PO 


Figure 95: Three pairs of non-crossing and one pair of crossing 
line segments. 


Plane-sweep algorithm. We obtain a faster algorithm 
by sweeping the plane with a vertical line from left to 
right, as before. To avoid special cases, we assume that 
no two endpoints are the same or lie on a common verti- 
cal line. During the sweep, we maintain the subset of line 
segments that intersect the sweep-line in the order they 
meet the line, as shown in Figure 96. We store this subset 


O 


Figure 96: Five of the line segments intersect the sweep-line at 
its current position and two of them cross. 


in a dictionary, which is updated at every endpoint. Only 
line segments that are adjacent in the ordering along the 
sweep-line are tested for crossings. Indeed, two line seg- 
ments that cross are adjacent right before the sweep-line 
passes through the crossing, if not earlier. 


Step 1. Sort the 2n endpoints from left to right and re- 
label them in this sequence as 71, %2,..., an. Each 
point still remembers the index of the other endpoint 
of its line segment. 


Step 2. For i from 1 to 2n, process the 7-th endpoint 
as follows: 


Case 2.1 2; is left endpoint of the line segment 
Tix; Therefore, i < j. Insert m;z; into 
the dictionary and let uv and pq be its prede- 
cessor and successor. If CROSS(u, v, £i, £j) 
or CROSS(p, q, £i, £j) then report the crossing 
and stop. 
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Case 2.2 2; is right endpoint of the line segment 
x;a;. Therefore, i > j. Let uv and pq be 
the predecessor and the successor of x;xj. If 
CROSS(u, v, p,q) then report the crossing and 
stop. Delete x;x,; from the dictionary. 


We do an insertion into the dictionary for each left end- 
point and a deletion from the dictionary for each right 
endpoint, both in time O(logn). In addition, we do at 
most two crossing tests per endpoint, which takes constant 
time. In total, the algorithm takes time O(n log n) to test 
whether a set of n line segments contains two that cross. 


21 Delaunay Triangulations 


The triangulations constructing by plane-sweep are typi- 
cally of inferior quality, that is, there are many long and 
skinny triangles and therefore many small and many large 
angles. We study Delaunay triangulations which distin- 
guish themselves from all other triangulations by a num- 
ber of nice properties, including they have fast algorithms 
and they avoid small angles to the extent possible. 


Plane-sweep versus Delaunay triangulation. Figures 
97 and 98 show two triangulations of the same set of 
points, one constructed by plane-sweep and the other the 
Delaunay triangulation. The angles in the Delaunay trian- 


Figure 97: Triangulation constructed by plane-sweep. Points on 
the same vertical line are processed from bottom to top. 


gulation seem consistently larger than those in the plane- 
sweep triangulation. This is not a coincidence and it can 
be proved that the Delaunay triangulation maximizes the 
minimum angle for every input set. Both triangulations 


Figure 98: Delaunay triangulation of the same twenty-one points 
triangulated in Figure 97. 


contain the edges that bound the convex hull of the input 
set. 
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Voronoi diagram. We introduce the Delaunay triangu- 
lation indirectly, by first defining a particular decomposi- 
tion of the plane into regions, one per point in the finite 
data set S. The region of the point u in S contains all 
points xv in the plane that are at least as close to u as to any 
other point in S, that is, 


Va = {reER?| |z—-ull < |z- vlv € S}, 


where ||x — ul] = [(z1 — u1)? + (£2 — u2)?]!/? is the Eu- 
clidean distance between the points x and u. We refer to 
Va as the Voronoi region of u. It is closed and its bound- 
ary consists of Voronoi edges which V, shares with neigh- 
boring Voronoi regions. A Voronoi edge ends in Voronoi 
vertices which it shares with other Voronoi edges. The 
Voronoi diagram of S is the collection of Voronoi regions, 
edges and vertices. Figure 99 illustrates the definitions. 
Let n be the number of points in S. We list some of the 
properties that will be important later. 


Figure 99: The (solid) Voronoi diagram drawn above the (dot- 
ted) Delaunay triangulation of the same twenty-one points trian- 
gulated in Figures 97 and 98. Some of the Voronoi edges are too 
far out to fit into the picture. 


e Each Voronoi region is a convex polygon constructed 
as the intersection of n — 1 closed half-planes. 


e The Voronoi region V, is bounded (finite) iff u lies in 
the interior of the convex hull of S. 


e The Voronoi regions have pairwise disjoint interiors 
and together cover the entire plane. 


Delaunay triangulation. We define the Delaunay trian- 
gulation as the straight-line dual of the Voronoi diagram. 
Specifically, for every pair of Voronoi regions V,, and V, 
that share an edge, we draw the line segment from u to v. 
By construction, every Voronoi vertex, x, has j > 3 clos- 
est input points. Usually there are exactly three closest 


points, u,v, w, in which case the triangle they span be- 
longs to the Delaunay triangulation. Note that x is equally 
far from u, v, and w and further from all other points in 
S. This implies the empty circle property of Delaunay tri- 
angles: all points of S — {u, v, w} lie outside the circum- 
scribed circle of www. Similarly, for each Delaunay edge 
uv, there is a circle that passes through u and v such that 
all points of S — {u,v} lie outside the circle. For exam- 
ple, the circle centered at the midpoint of the Voronoi edge 
shared by V,, and V, is empty in this sense. This property 
can be used to prove that the edge skeleton of the Delau- 
nay triangulation is a straight-line embedding of a planar 
graph. 


Figure 100: A Voronoi vertex of degree 5 and the corresponding 
pentagon in the Delaunay triangulation. The dotted edges com- 
plete the triangulation by decomposing the pentagon into three 
triangles. 


Now suppose there is a vertex with degree j > 3. It cor- 
responds to a polygon with 7 > 3 edges in the Delaunay 
triangulation, as illustrated in Figure 100. Strictly speak- 
ing, the Delaunay triangulation is no longer a triangulation 
but we can complete it to a triangulation by decompos- 
ing each j-gon into j — 2 triangles. This corresponds to 
perturbing the data points every so slightly such that the 
degree-7 Voronoi vertices are resolved into trees in which 
j — 2 degree-3 vertices are connected by 7 — 3 tiny edges. 


Local Delaunayhood. Given a triangulation of a finite 
point set S, we can test whether or not it is the Delaunay 
triangulation by testing each edge against the two trian- 
gles that share the edge. Suppose the edge wv in the tri- 
angulation T is shared by the triangles uvp and uvq. We 
call uv locally Delaunay, or ID for short, if q lies on or 
outside the circle that passes through u,v, p. The condi- 
tion is symmetric in p and q because the circle that passes 
through u, v, p intersects the first circle in points u and v. 
It follows that p lies on or outside the circle of u, v, q iff q 
lies on or outside the circle of u, v, p. We also call wv lo- 
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cally Delaunay if it bounds the convex hull of S and thus 
belongs to only one triangle. The local condition on the 
edges implies a global property. 


DELAUNAY LEMMA. If every edge in a triangulation K 
of S is locally Delaunay then K is the Delaunay tri- 
angulation of S. 


Although every edge of the Delaunay triangulation is lo- 
cally Delaunay, the Delaunay Lemma is not trivial. In- 
deed, K may contain edges that are locally Delaunay but 
do not belong to the Delaunay triangulation, as shown in 
Figure 101. We omit the proof of the lemma. 


Figure 101: The edge wv is locally Delaunay but does not belong 
to the Delaunay triangulation. 


Edge-flipping. The Delaunay Lemma suggests we con- 
struct the Delaunay triangulation by first constructing an 
arbitrary triangulation of the point set S and then modify- 
ing it locally to make all edges 1D. The idea is to look for 
non-ID edges and to flip them, as illustrated in Figure 102. 
Indeed, if uv is a non-ID edge shared by triangles uvp and 


p 


Figure 102: The edge wv is non-ID and can be flipped to the edge 
pq, which is ID. 


uvq then upvq is a convex quadrilateral and flipping uv 
means substituting one diagonal for the other, namely pq 


for uv. Note that if wv is non-ID then pq is ID. It is im- 
portant that the algorithm finds non-ID edges quickly. For 
this purpose, we use a stack of edges. Initially, we push 
all edges on the stack and mark them. 


while stack is non-empty do 
pop edge uv from stack and unmark it; 
if uv is non-ID then 
substitute pq for uv; 
for ab € {up, pu, vq, qu} do 
if abis unmarked then 
push ab on the stack and mark it 
endif 
endfor 
endif 
endwhile. 


The marks avoid multiple copies of the same edge on the 
stack. This implies that at any one moment the size of the 
stack is less than 3n. Note also that initially the stack con- 
tains all non-ID edges and that this property is maintained 
as an invariant of the algorithm. The Delaunay Lemma 
implies that when the algorithm halts, which is when the 
stack is empty, then the triangulation is the Delaunay tri- 
angulation. However, it is not yet clear that the algorithm 
terminates. Indeed, the stack can grow and shrink dur- 
ing the course of the algorithm, which makes it difficult to 
prove that it ever runs empty. 


In-circle test. Before studying the termination of the al- 
gorithm, we look into the question of distinguishing 1D 
from non-ID edges. As before we assume that the edge uv 
is shared by the triangles uvp and uvq in the current trian- 
gulation. Recall that wv is ID iff q lies outside the circle 
that passes through u, v, p. Let f : R? — R be defined by 
f(x) = x? + x2. As illustrated in Figure 103, the graph 
of this function is a paraboloid in three-dimensional space 
and we write xt = (21,22, f(x)) for the vertical projec- 
tion of the point x onto the paraboloid. Assuming the three 
points u, v, p do not lie on a common line then the points 
ut,ut,p* lie on a non-vertical plane that is the graph of 
a function h(x) = axı + Bx2 + y. The projection of the 
intersection of the paraboloid and the plane back into R? 
is given by 


0 = f 


T 


(x) — h(x) 


Ba 


ary ue 


which is the equation of a circle. This circle passes 
through u,v, p so it is the circle we have to compare q 
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+ 


+ vut intersects the 


Figure 103: The plane passing through u p 
paraboloid in an ellipse whose projection into R? passes through 
the points u,v, p. The point q* lies below the plane iff q lies 


inside the circle. 


against. We note that q lies inside the circle iff gt lies be- 
low the plane. The latter test can be based on the sign of 
the determinant of the 4-by-4 matrix 


1 u uw u? +u? 

1 v vw v+? 

A = 1 2 p 
Pı P2 Pi TPs 

ln e +É 


Exchanging two rows in the matrix changes the sign. 
While the in-circle test should be insensitive to the order 
of the first three points, the sign of the determinant is not. 
We correct the change using the sign of the determinant of 
the 3-by-3 matrix that keeps track of the ordering of u, v, p 
along the circle, 


1 ui U2 
T = Uy v2 
1 pi pe 


Now we claim that s is inside the circle of u, v, p iff the 
two determinants have opposite signs: 


boolean INCIRCLE(Points u,v, p,q) 
return detr -det A < 0. 


We first show that the boolean function is correct for u = 
(0,0), v = (1,0), p = (0,1), and q = (0,0.5). The sign 
of the product of determinants remains unchanged if we 
continuously move the points and avoid the configurations 
that make either determinant zero, which are when u, v, p 
are collinear and when u,v, p,q are cocircular. We can 
change any configuration where q is inside the circle of 
u, v, p continuously into the special configuration without 
going through zero, which implies the correctness of the 
function for general input points. 


Termination and running time. To prove the edge-flip 
algorithm terminates, we imagine the triangulation lifted 
to R°. We do this by projecting the vertices vertically 
onto the paraboloid, as before, and connecting them with 
straight edges and triangles in space. Let wv be an edge 
shared by triangles uvp and uvq that is flipped to pq by 
the algorithm. It follows the line segments uv and pq cross 
and their endpoints form a convex quadrilateral, as shown 
in Figure 104. After lifting the two line segments, we get 


Figure 104: A flip in the plane lifts to a tetrahedron in space in 
which the ID edge passes below the non-ID edge. 


u*v™ passing above ptq*. We may thus think of the flip 
as gluing the tetrahedron u*v* ptq* underneath the sur- 
face obtained by lifting the triangulation. The surface is 
pushed down by each flip and never pushed back up. The 
removed edge is now above the new surface and can there- 
fore not be reintroduced by a later flip. It follows that the 
algorithm performs at most (3) flips and thus takes at most 
time O(n?) to construct the Delaunay triangulation of S. 
There are faster algorithms that work in time O(n log n) 
but we prefer the suboptimal method because it is simpler 
and it reveals more about Delaunay triangulations than the 
other algorithms. 


The lifting of the input points to R? leads to an interest- 
ing interpretation of the edge-flip algorithm. Starting with 
a monotone triangulated surface passing through the lifted 
points, we glue tetrahedra below the surface until we reach 
the unique convex surface that passes through the points. 
The projection of this convex surface is the Delaunay tri- 
angulation of the points in the plane. This also gives a 
reinterpretation of the Delaunay Lemma in terms of con- 
vex and concave edges of the surface. 
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22 Alpha Shapes 


Many practical applications of geometry have to do with 
the intuitive but vague concept of the shape of a finite point 
set. To make this idea concrete, we use the distances be- 
tween the points to identify subcomplexes of the Delaunay 
triangulation that represent that shape at different levels of 
resolution. 


Union of disks. Let S be a set of n points in R?. For 
each r > 0, we write B,(r) = {x € R? | |z- ul] < 
r} for the closed disk with center u and radius r. Let 
U(r) = Uaes Bu(r) be the union of the n disks. We de- 
compose this union into convex sets of the form R,,(r) = 
Ba(r) O Va. Then 


(i) Ra(r) is closed and convex for every point u € S 
and every radius r > 0; 


(ii) Ra(r) and R,(r) have disjoint interiors whenever the 
two points, u and v, are different; 


üi U(r) = Uues Ru(r)- 


We illustrate this decomposition in Figure 105. Each re- 
gion R,,(r) is the intersection of n — 1 closed half-planes 
and a closed disk. All these sets are closed and convex, 
which implies (i). The Voronoi regions have disjoint inte- 
riors, which implies (ii). Finally, take a point x € U(r) 
and let u be a point in S with x € V,. Then x € B,(r) 
and therefore x € R,,(a). This implies (iii). 


a J- 


aa oS | 
Figure 105: The Voronoi decomposition of a union of eight disks 
in the plane and superimposed dual alpha complex. 


Nerve. Similar to defining the Delaunay triangulation as 
the dual of the Voronoi diagram, we define the alpha com- 
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plex as the dual of the Voronoi decomposition of the union 
of disks. This time around, we do this more formally. Let- 
ting C be a finite collection of sets, the nerve of C is the 
system of subcollections that have a non-empty common 
intersection, 
NrvC = {XCC|()X #0} 

This is an abstract simplicial complex since (| X 4 @ and 
Y C X implies (| Y 4 @. For example, if C is the collec- 
tion of Voronoi regions then Nrv C is an abstract version 
of the Delaunay triangulation. More specifically, this is 
true provide the points are in general position and in par- 
ticular no four points lie on a common circle. We will as- 
sume this for the remainder of this section. We say the De- 
launay triangulation is a geometric realization of Nrv C, 
namely the one obtained by mapping each Voronoi region 
(a vertex in the abstract simplicial complex) to the gener- 
ating point. All edges and triangles are just convex hulls 
of their incident vertices. To go from the Delaunay trian- 
gulation to the alpha complex, we substitute the regions 
R.(r) for the V,,. Specifically, 

Alpha(r) = Nrv{R,(r) | ue S}. 
Clearly, this is isomorphic to a subcomplex of the nerve 
of Voronoi regions. We can therefore draw Alpha(r) as 
a subcomplex of the Delaunay triangulation; see Figure 
105. We call this geometric realization of Alpha(r) the 
alpha complex for radius r, denoted as A(r). The alpha 
shape for the same radius is the underlying space of the 
alpha complex, | A(r)|. 


The nerve preserves the way the union is connected. 
In particular, their Betti numbers are the same, that is, 
B,(U(r)) = B,(A(r)) for all dimensions p and all radii 
r. This implies that the union and the alpha shape have 
the same number of components and the same number of 
holes. For example, in Figure 105 both have one compo- 
nent and two holes. We omit the proof of this property. 


Filtration. We are interested in the sequence of alpha 
shapes as the radius grows from zero to infinity. Since 
growing r grows the regions R,„(r), the nerve can only 
get bigger. In other words, A(r) C A(s) whenever r < s. 
There are only finitely many subcomplexes of the Delau- 
nay triangulation. Hence, we get a finite sequence of alpha 
complexes. Writing A; for the i-th alpha complex, we get 
the following nested sequence, 


S=A,C Aoc...CAp=D 


where D denotes the Delaunay triangulation of S. We 
call such a sequence of complexes a filtration. We illus- 
trate this construction in Figure 106. The sequence of al- 


Figure 106: A finite sequence of unions of disks, all decomposed 
by the same Voronoi diagram. 


pha complexes begins with a set of n isolated vertices, the 
points in S. To go from one complex to the next, we either 
add an edge, we add a triangle, or we add a pair consisting 
of a triangle with one of its edges. In Figure 106, we be- 
gin with eight vertices and get the following sequence of 
alpha complexes. 


A, = {a,b,c,d,e,f,g,h}; 
A = A,U {ah}; 

A3 = AU {be}; 

A, = As3U {ab, ef}; 

Ax = AU {de}; 

Ag = AsU{gh}; 

A7 = A6U {cd}; 

Ag = ArU {fg}; 

Ag = AgU {cg}. 


Going from A7 to Ag, we get for the first time a 1-cycle, 
which bounds a hole in the embedding. In Ag, this hole is 
cut into two. This is the alpha complex depicted in Figure 
105. We continue. 


Avo = AgU {cf}; 

Ai, = AU {abh, bh}; 
Aı2 = AU {cde, ce}; 
Aiz = AU {cfg}; 
Ai = Aijs3U {cef}; 
Ais = Aj4U {bch, ch}; 
Aig = Ais U {cgh}. 
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At this moment, we have a triangulated disk but not yet the 
entire Delaunay triangulation since the triangle bcd and the 
edge bd are still missing. Each step is generic except when 
we add two equally long edges to A3. 


Compatible ordering of simplices. We can represent 
the entire filtration of alpha complexes compactly by sort- 
ing the simplices in the order they join the growing com- 
plex. An ordering 01, 02,...,0m of the Delaunay sim- 
plices is compatible with the filtration if 


1. the simplices in A; precede the ones not in A; for 
each 2; 


2. the faces of a simplex precede the simplex. 


For example, the sequence 


a,b,c, d,e, f, g, h; ah; be; ab, ef; 
de; gh; cd; fg; cg; cf; bh, abh; ce, 
cde; cfg; cef ; ch, bch; cgh; bd; bed 


is compatible with the filtration in Figure 106. Every alpha 
complex is a prefix of the compatible sequence but not 
necessarily the other way round. Condition 2 guarantees 
that every prefix is a complex, whether an alpha complex 
or not. We thus get a finer filtration of complexes 


P=Kyp CKic...C Kn =D, 


where K; is the set of simplices from g1 to o;. To con- 
struct the compatible ordering, we just need to compute 
for each Delaunay simplex the radius r; = r(o;) such that 
oi E€ A(r) iff r > ri. For a vertex, this radius is zero. 
For a triangle, this is the radius of the circumcircle. For 


Figure 107: Left: the middle edge belongs to two acute triangles. 
Right: it belongs to an obtuse and an acute triangle. 


an edge, we have two cases. Let y and w be the angles 
opposite the edge g; inside the two incident triangles. We 
have y + w > 180° because of the empty circle property. 


CASE 1. p < 90° and w < 90°. Then r; = r(o;) is half 
the length of the edge. 


CASE 2. » > 90°. Then r; = rj, where gj is the incident 
triangle with angle y. 


Both cases are illustrated in Figure 107. In Case 2, the 
edge g; enters the growing alpha complex together with 
the triangle oj. The total number of simplices in the De- 
launay triangulation is m < 6n. The threshold radii can 
be computed in time O(n). Sorting the simplices into 
the compatible ordering can therefore be done in time 
O(n logn). 


Betti numbers. In two dimensions, Betti numbers can 
be computed directly, without resorting to boundary matri- 
ces. The only two possibly non-zero Betti numbers are 6o, 
the number of components, and 61, the number of holes. 
We compute the Betti numbers of K; by adding the sim- 
plices in order. 


Bo = Pi = 0; 
fori=1tojgdo 
switch dimo;: 
case 0: Bo = bo +1; 
case 1: let u, v be the endpoints of o;; 
if FIND(u) = FIND(v) then bı = 6, +1 
else fo = fo — 1; 
UNION(u, v) 
endif 
case 2: 61 = 6, — 1 
endswitch 
endfor. 


All we need is tell apart the two cases when o; is an edge. 
This is done using a union-find data structure maintaining 
the components of the alpha complex in amortized time 
a(n) per simplex. The total running time of the algorithm 
for computing Betti numbers is therefore O(na(n)). 
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Sixth Homework Assignment 


Write the solution to each problem on a single page. The 
deadline for handing in solutions is November 25. 


Problem 1. (20 points). Let S be a set of n unit disks 
in the Euclidean plane, each given by its center and 
radius, which is one. Give an algorithm that decides 
whether any two of the disks in S intersect. 


Problem 2. (20 = 10+ 10 points). Let S be a set of 
n points in the Euclidean plane. The Gabriel graph 
connects points u,v € S with a straight edge if 


lull? < le- pl? + le- pl? 


for every point p in S. 


(a) Show that the Grabriel graph is a subgraph of 
the edge skeleton of the Delaunay triangulation. 

(b) Is the Gabriel graph necessarily connected? 
Justify your answer. 


Problem 3. (20 = 10 + 10 points). Consider a set of n > 
3 closed disks in the Euclidean plane. The disks are 
allowed to touch but no two of them have an interior 
point in common. 


(a) Show that the number of touching pairs of disks 
is at most 3n — 6. 

(b) Give a construction that achieves the upper 
bound in (a) for any n > 3. 


Problem 4. (20 = 10 + 10 points). Let K be a triangula- 
tion of a set of n > 3 points in the plane. Let L be a 
line that avoids all the points. 


(a) Prove that L intersects at most 2n — 4 of the 
edges in K. 

(b) Give a construction for which L achieves the 
upper bound in (a) for any n > 3. 


Problem 5. (20 points). Let S' be a set of n points in the 
Euclidean plane, consider its Delaunay triangulation 
and the corresponding filtration of alpha complexes, 


S = A, CAC... C Ap. 


Under what conditions is it true that A; and A;+1 dif- 
fer by a single simplex for every 1 < i < m — 1? 
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23 Easy and Hard Problems 


The theory of NP-completeness is an attempt to draw a 
line between tractable and intractable problems. The most 
important question is whether there is indeed a difference 
between the two, and this question is still unanswered. 
Typical results are therefore relative statements such as “if 
problem B has a polynomial-time algorithm then so does 
problem C” and its equivalent contra-positive “if prob- 
lem C has no polynomial-time algorithm then neither has 
problem B”. The second formulation suggests we remem- 
ber hard problems C and for a new problem B we first see 
whether we can prove the implication. If we can then we 
may not want to even try to solve problem B efficiently. A 
good deal of formalism is necessary for a proper descrip- 
tion of results of this kind, of which we will introduce only 
a modest amount. 


What is a problem? An abstract decision problem is a 
function I — {0,1}, where J is the set of problem in- 
stances and 0 and 1 are interpreted to mean FALSE and 
TRUE, as usual. To completely formalize the notion, we 
encode the problem instances in strings of zeros and ones: 
I — {0,1}*. A concrete decision problem is then a func- 
tion Q : {0,1}* — {0,1}. Following the usual conven- 
tion, we map bit-strings that do not correspond to mean- 
ingful problem instances to 0. 


As an example consider the shortest-path problem. A 
problem instance is a graph and a pair of vertices, u and 
v, in the graph. A solution is a shortest path from u and 
v, or the length of such a path. The decision problem ver- 
sion specifies an integer k and asks whether or not there 
exists a path from u to v whose length is at most k. The 
theory of NP-completeness really only deals with deci- 
sion problems. Although this is a loss of generality, the 
loss is not dramatic. For example, given an algorithm for 
the decision version of the shortest-path problem, we can 
determine the length of the shortest path by repeated de- 
cisions for different values of k. Decision problems are 
always easier (or at least not harder) than the correspond- 
ing optimization problems. So in order to prove that an 
optimization problem is hard it suffices to prove that the 
corresponding decision problem is hard. 


Polynomial time. An algorithm solves a concrete deci- 
sion problem Q in time T(n) if for every instance x € 
{0,1}* of length n the algorithm produces Q(x) in time 
at most T(n). Note that this is the worst-case notion of 
time-complexity. The problem Q is polynomial-time solv- 
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able if T(n) = O(n*) for some constant k independent of 
n. The first important complexity class of problems is 


P = set of concrete decision problems 


that are polynomial-time solvable. 


The problems Q € P are called tractable or easy and the 
problems Q ¢ P are called intractable or hard. Algo- 
rithms that take only polynomial time are called efficient 
and algorithms that require more than polynomial time 
are inefficient. In other words, until now in this course 
we only talked about efficient algorithms and about easy 
problems. This terminology is adapted because the rather 
fine grained classification of algorithms by complexity we 
practiced until now is not very useful in gaining insights 
into the rather coarse distinction between polynomial and 
non-polynomial. 


It is convenient to recast the scenario in a formal lan- 
guage framework. A language is a set L C {0,1}*. We 
can think of it as the set of problem instances, x, that 
have an affirmative answer, Q(x) = 1. An algorithm 
A: {0,1}* — {0,1} accepts x € {0,1}* if A(x) = 1 
and it rejects x if A(x) = 0. The language accepted by A 
is the set of strings x € {0,1}* with A(x) = 1. There is 
a subtle difference between accepting and deciding a lan- 
guage L. The latter means that A accepts every x € L and 
rejects every x ¢ L. For example, there is an algorithm 
that accepts every program that halts, but there is no algo- 
rithm that decides the language of such programs. Within 
the formal language framework we redefine the class of 
polynomial-time solvable problems as 


P = {LC {0,1}* | Lis accepted by 


a polynomial-time algorithm} 
= {LC {0,1}* | Lis decided by 


a polynomial-time algorithm}. 


Indeed, a language that can be accepted in polynomial 
time can also be decided in polynomial time: we keep 
track of the time and if too much goes by without x be- 
ing accepted, we turn around and reject x. This is a non- 
constructive argument since we may not know the con- 
stants in the polynomial. However, we know such con- 
stants exist which suffices to show that a simulation as 
sketched exists. 


Hamiltonian cycles. We use a specific graph problem to 
introduce the notion of verifying a solution to a problem, 
as opposed to solving it. Let Œ = (V, E) be an undi- 
rected graph. A hamiltonian cycle contains every vertex 


v € V exactly once. The graph G is hamiltonian if it has 
a hamiltonian cycle. Figure 108 shows a hamiltonian cy- 
cle of the edge graph of a Platonic solid. How about the 
edge graphs of the other four Platonic solids? Define L = 


Figure 108: The edge graph of the dodecahedron and one of its 
hamiltonian cycles. 


{G | Gis hamiltonian}. We can thus ask whether or not 
L € P, that is, whether or not there is a polynomial-time 
algorithm that decides whether or not a graph is hamilto- 
nian. The answer to this question is currently not known, 
but there is evidence that the answer might be negative. On 
the other hand, suppose y is a hamiltonian cycle of G. The 
language L’ = {(G, y) | y is a hamiltonian cycle of G} is 
certainly in P because we just need to make sure that y 
and G have the same number of vertices and every edge of 
y is also an edge of G. 


Non-deterministic polynomial time. More generally, it 
seems easier to verify a given solution than to come up 
with one. In a nutshell, this is what NP-completeness is 
about, namely finding out whether this is indeed the case 
and whether the difference between accepting and verify- 
ing can be used to separate hard from easy problems. 


Call y € {0,1}* a certificate. An algorithm A verifies 

a problem instance x € {0, 1}* if there exists a certificate 
y with A(x, y) = 1. The language verified by A is the set 
of strings x € {0, 1}* verified by A. We now define a new 
class of problems, 
NP = {LC {0,1}* | Lis verified by 


a polynomial-time algorithm}. 


More formally, L is in NP if for every problem instance 
x € L there is a certificate y whose length is bounded 
from above by a polynomial in the length of x such that 
A(x,y) = 1 and A runs in polynomial time. For exam- 
ple, deciding whether or not G is hamiltonian is in NP. 
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The name NP is an abbreviation for non-deterministic 
polynomial time, because a non-deterministic computer 
can guess a certificate and then verify that certificate. In a 
parallel emulation, the computer would generate all possi- 
ble certificates and then verify them in parallel. Generat- 
ing one certificate is easy, because it only has polynomial 
length, but generating all of them is hard, because there 
are exponentially many strings of polynomial length. 


Figure 109: Four possible relations between the complexity 
classes P, NP, and co-NP. 


Non-deterministic machine are at least as powerful as 
deterministic machines. It follows that every problem in 
P is also in NP, P C NP. Define 

co-NP = {L|L={xrg L}e€ NP}, 
which is the class of languages whose complement can 
be verified in non-deterministic polynomial time. It is 
not known whether or not NP = co-NP. For example, 
it seems easy to verify that a graph is hamiltonian but 
it seems hard to verify that a graph is not hamiltonian. 
We said earlier that if L € P then Z € P. Therefore, 
P C co-NP. Hence, only the four relationships between 
the three complexity classes shown in Figure 109 are pos- 
sible, but at this time we do not know which one is correct. 


Problem reduction. We now develop the concept of re- 
ducing one problem to another, which is key in the con- 
struction of the class of NP-complete problems. The idea 
is to map or transform an instance of a first problem to an 
instance of a second problem and to map the solution to 
the second problem back to a solution to the first problem. 
For decision problems, the solutions are the same and need 
no transformation. 


Language Lı is polynomial-time reducible to language 
Lz, denoted Lı <p Le, if there is a polynomial-time com- 
putable function f : {0,1}* — {0,1}* such that z € Ly 
iff f(x) € Lo, for all x € {0,1}*. Now suppose that 


Lı is polynomial-time reducible to Lə and that Lə has a 
polynomial-time algorithm A» that decides Do, 


a + f(x) £> {0,1}. 


We can compose the two algorithms and obtain a poly- 
nomial-time algorithm A; = Ag o f that decides Lı. In 
other words, we gained an efficient algorithm for Lı just 
by reducing it to Lə. 


REDUCTION LEMMA. If Lı <p Lə and Le € P then 
Lı €P. 


In words, if Lı is polynomial-time reducible to Lə and 
Lə is easy then L; is also easy. Conversely, if we know 
that Lı is hard then we can conclude that Lə is also hard. 
This motivates the following definition. A language L C 
{0, 1}* is NP-complete if 


(1) L € NP; 
(2) L’ <p L, for every L’ € NP. 


Since every L’ € NP is polynomial-time reducible to L, 
all L’ have to be easy for L to have a chance to be easy. 
The L’ thus only provide evidence that L might indeed 
be hard. We say L is NP-hard if it satisfies (2) but not 
necessarily (1). The problems that satisfy (1) and (2) form 
the complexity class 


NPG = {L| Lis NP-complete}. 

All these definitions would not mean much if we could 
not find any problems in NPC. The first step is the most 
difficult one. Once we have one problem in NPC we can 
get others using reductions. 


Satisfying boolean formulas. Perhaps surprisingly, a 
first NP-complete problem has been found, namely the 
problem of satisfiability for logical expressions. A 
boolean formula, p, consists of variables, £1, £2, ..., Op- 
erators, =, ^, V,==>,..., and parentheses. A truth assign- 
ment maps each variable to a boolean value, 0 or 1. The 
truth assignment satisfies if the formula evaluates to 1. The 
formula is satisfiable if there exists a satisfying truth as- 
signment. Define SAT = {y | wis satisfiable}. As an 
example consider the formula 


wp (xı 


If we set 21 = £2 = 1 we get (z1 => v2) = 1, (£2 V 
~xı) = 1 and therefore y = 1. It follows that Y € SAT. 


z2) (£2 V 01). 
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In fact, all truth assignments evaluate to 1, which means 
that ~ is really a tautology. More generally, a boolean 
formula, y, is satisfyable iff ~ọ is not a tautology. 


SATISFIABILITY THEOREM. We have SAT € NP and 
L’ <p SAT for every L’ € NP. 


That SAT is in the class NP is easy to prove: just guess an 
assignment and verify that it satisfies. However, to prove 
that every L’ € NP can be reduced to SAT in polynomial 
time is quite technical and we omit the proof. The main 
idea is to use the polynomial-time algorithm that verifies 
L’ and to construct a boolean formula from this algorithm. 
To formalize this idea, we would need a formal model of a 
computer, a Touring machine, which is beyond the scope 
of this course. 


24 NP-Complete Problems 


In this section, we discuss a number of NP-complete prob- 
lems, with the goal to develop a feeling for what hard 
problems look like. Recognizing hard problems is an im- 
portant aspect of a reliable judgement for the difficulty of 
a problem and the most promising approach to a solution. 
Of course, for NP-complete problems, it seems futile to 
work toward polynomial-time algorithms and instead we 
would focus on finding approximations or circumventing 
the problems altogether. We begin with a result on differ- 
ent ways to write boolean formulas. 


Reduction to 3-satisfiability. We call a boolean vari- 
able or its negation a literal. The conjunctive normal 
form is a sequence of clauses connected by A^s, and each 
clause is a sequence of literals connected by Vs. A for- 
mula is in 3-CNF if it is in conjunctive normal form and 
each clause consists of three literals. It turns out that de- 
ciding the satisfiability of a boolean formula in 3-CNF 
is no easier than for a general boolean formula. Define 
3-SAT = {y € SAT | vis in 3-CNF}. We prove the 
above claim by reducing SAT to 3-SAT. 


SATISFIABILITY LEMMA. SAT <p 3-SAT. 


PROOF. We take a boolean formula y and transform it into 
3-CNF in three steps. 


Step 1. Think of yas an expression and represent it as 
a binary tree. Each node is an operation that gets the 
input from its two children and forwards the output 
to its parent. Introduce a new variable for the output 
and define a new formula y’ for each node, relating 
the two input edges with the one output edge. Figure 
110 shows the tree representation of the formula y = 
(zı T2) (aq V 72:1). The new formula is 

f 


p (y2 (£1 x2)) 
A(y3 => (x2 V 721)) 
Aly => (y2 => y3)) A y1- 


It should be clear that there is a satisfying assignment 
for ọ iff there is one for y’. 


Step 2. Convert each clause into disjunctive normal 
form. The most mechanical way uses the truth table 
for each clause, as illustrated in Table 6. Each clause 
has at most three literals. For example, the negation 
of yo <> (xı = > 2) is equivalent to the disjunc- 
tion of the conjunctions in the rightmost column. It 
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Figure 110: The tree representation of the formula y. Inciden- 
tally, y is a tautology, which means it is satisfied by every truth 
assignment. Equivalently, ~y is not satisfiable. 


prohibited 
ay2 A 721 A 7%2 
ay2 A 721 A £2 


< 
8 


ay2 A z1 A x2 


y2 ATIA 722 


0 
0 
0 
0 
1 
1 
1 
1 


FPOrFRrFROHFO 


Table 6: Conversion of a clause into a disjunction of conjunctions 
of at most three literals each. 


follows that yo 4= (zı => 2) is equivalent to the 
negation of that disjunction, which by de Morgan’s 
law is (y2 V £1 V £2) A (Y2 V z1 V 702) A (y2 V azı V 
22) A (“y2 V 7x1 V £2). 


Step 3. The clauses with fewer than three literals can 
be expanded by adding new variables. For example 
a V bis expanded to (a V b V p) A (a V b V 7p) and 
(a) is expanded to (a V p V q) A (a V pV ~q) A (a V 
=p V q) ^ (a V =p V =q). 


Each step takes only polynomial time. At the end, we get 
an equivalent formula in 3-conjunctive normal form. 


We note that clauses of length three are necessary to 
make the satisfiability problem hard. Indeed, there is a 
polynomial-time algorithm that decides the satisfiability 
of a formula in 2-CNF. 


NP-completeness proofs. Using polynomial-time re- 
ductions, we can show fairly mechanically that problems 
are NP-complete, if they are. A key property is the tran- 
sitivity of <p, that is, if L’ <p Lı and Ly <p Lə 
then L’ <p Lə, as can be seen by composing the two 
polynomial-time computable functions to get a third one. 


REDUCTION LEMMA. Let Lı, Lə C {0,1}* and assume 
Lı <p Lə. If Lı is NP-hard and Ly € NP then 
La € NPC. 


A generic NP-completeness proof thus follows the steps 
outline below. 


Step 1. Prove that Lə € NP. 


Step 2. Selecta known NP-hard problem, Lı, and find 
a polynomial-time computable function, f, with x € 
Li iff f(z) € Lə. 


This is what we did for Lə = 3-SAT and Lı = SAT. 
Therefore 3-SAT € NPC. Currently, there are thousands 
of problems known to be NP-complete. This is often con- 


Figure 111: Possible relation between P, NPC, and NP. 


sidered evidence that P 4 NP, which can be the case only 
if P A NPC = @, as drawn in Figure 111. 


Cliques and independent sets. There are many NP- 
complete problems on graphs. A typical such problem 
asks for the largest complete subgraph. Define a clique 
in an undirected graph G = (V, Æ) as a subgraph (W, F`) 
with F = (X): Given G and an integer k, the CLIQUE 
problem asks whether or not there is a clique of k or more 
vertices. 


CLAIM. CLIQUE € NPC. 


PROOF. Given k vertices in G, we can verify in poly- 
nomial time whether or not they form a complete graph. 
Thus CLIQUE € NP. To prove property (2), we show 
that 3-SAT <p CLIQUE. Let vy be a boolean formula in 
3-CNF consisting of k clauses. We construct a graph as 
follows: 


(i) each clause is replaced by three vertices; 


(ii) two vertices are connected by an edge if they do not 
belong to the same clause and they are not negations 
of each other. 


In a satisfying truth assignment, there is at least one true 
literal in each clause. The true literals form a clique. Con- 
versely, a clique of k or more vertices covers all clauses 
and thus implies a satisfying truth assignment. 
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It is easy to decide in time O(k?n**?) whether or not a 
graph of n vertices has a clique of size k. If k is a constant, 
the running time of this algorithm is polynomial in n. For 
the CLIQUE problem to be NP-complete it is therefore es- 
sential that k be a variable that can be arbitrarily large. 
We use the NP-completeness of finding large cliques to 
prove the NP-completeness of large sets of pairwise non- 
adjacent vertices. Let G = (V, E) be an undirected graph. 
A subset W C V is independent if none of the vertices in 
W are adjacent or, equivalently, if Æ N (es = (). Given 
G and an integer k, the INDEPENDENT SET problem asks 
whether or not there is an independent set of k or more 
vertices. 


CLAIM. INDEPENDENT SET € NPC. 
PROOF. It is easy to verify that there is an independent set 


of size k: just guess a subset of k vertices and verify that 
no two are adjacent. 


Figure 112: The four shaded vertices form an independent set in 
the graph on the left and a clique in the complement graph on the 
right. 


We complete the proof by reducing the CLIQUE to the 
INDEPENDENT SET problem. As illustrated in Figure 112, 
W C V is independent iff W defines a clique in the com- 
plement graph, G = (V, (5) — E). To prove CLIQUE <p 
INDEPENDENT SET, we transform an instance H, k of the 
CLIQUE problem to the instance G = H, k of the INDE- 
PENDENT SET problem. G has an independent set of size 
k or larger iff H has a clique of size k or larger. 


Various NP-complete graph problems. We now de- 
scribe a few NP-complete problems for graphs without 
proving that they are indeed NP-complete. Let G = 
(V, E) be an undirected graph with n vertices and k a pos- 
itive integer, as before. The following problems defined 
for G and k are NP-complete. 


An ¢-coloring of G is a function y : V — [é] with 
x(u) # x(v) whenever u and v are adjacent. The CHRO- 
MATIC NUMBER problem asks whether or not G has an £- 
coloring with £ < k. The problem remains NP-complete 


for fixed k > 3. For k = 2, the CHROMATIC NUMBER 
problem asks whether or not G is bipartite, for which there 
is a polynomial-time algorithm. 


The bandwidth of G is the minimum £ such that there 
is a bijection 6 : V — [n] with |G(u) — B(v)| < £ for 
all adjacent vertices u and v. The BANDWIDTH problem 
asks whether or not the bandwidth of G is k or less. The 
problem arises in linear algebra, where we permute rows 
and columns of a matrix to move all non-zero elements of 
a square matrix as close to the diagonal as possible. For 
example, if the graph is a simple path then the bandwidth 
is 1, as can be seen in Figure 113. We can transform the 


Figure 113: Simple path and adjacency matrix with rows and 
columns ordered along the path. 


adjacency matrix of G such that all non-zero diagonals are 
at most the bandwidth of G away from the main diagonal. 


Assume now that the graph G is complete, E = 
(X); and that each edge, uv, has a positive integer 
weight, w(uv). The TRAVELING SALESMAN problem 
asks whether there is a permutation uo, U1,...,Un—1 Of 
the vertices such that the sum of edges connecting con- 
tiguous vertices (and the last vertex to the first) is k or 
less, 


Ww(Uiui+i) < k 


b 


Il 
(æ 


where indices are taken modulo n. The problem remains 
NP-complete if w : E — {1,2} (reduction to HAMILTO- 
NIAN CYCLE problem), and also if the vertices are points 
in the plane and the weight of an edge is the Euclidean 
distance between the two endpoints. 


Set systems. Simple graphs are set systems in which the 
sets contain only two elements. We now list a few NP- 
complete problems for more general set systems. Letting 
V be a finite set, C C 2V a set system, and k a positive 
integer, the following problems are NP-complete. 


The PACKING problem asks whether or not C has k or 
more mutually disjoint sets. The problem remains NP- 
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complete if no set in C contains more than three elements, 
and there is a polynomial-time algorithm if every set con- 
tains two elements. In the latter case, the set system is a 
graph and a maximum packing is a maximum matching. 


The COVERING problem asks whether or not C' has k 
or fewer subsets whose union is V. The problem remains 
NP-complete if no set in C contains more than three ele- 
ments, and there is a polynomial-time algorithm if every 
sets contains two elements. In the latter case, the set sys- 
tem is a graph and the minimum cover can be constructed 
in polynomial time from a maximum matching. 


Suppose every element v € V has a positive integer 
weight, w(v). The PARTITION problem asks whether 
there is a subset U C V with 


X vwu) = 


ucU 


5 w(v). 


veEeV—U 


The problem remains NP-complete if we require that U 
and V — U have the same number of elements. 


25 Approximation Algorithms 


Many important problems are NP-hard and just ignoring 
them is not an option. There are indeed many things one 
can do. For problems of small size, even exponential- 
time algorithms can be effective and special subclasses 
of hard problems sometimes have polynomial-time algo- 
rithms. We consider a third coping strategy appropriate 
for optimization problems, which is computing almost op- 
timal solutions in polynomial time. In case the aim is 
to maximize a positive cost, a o(n)-approximation algo- 
rithm is one that guarantees to find a solution with cost 
C > C*/o(n), where C* is the maximum cost. For mini- 
mization problems, we would require C < C*o(n). Note 
that o(n) > 1 and if o(n) = 1 then the algorithm produces 
optimal solutions. Ideally, ọ is a constant but sometime 
even this is not achievable in polynomial time. 


Vertex cover. The first problem we consider is finding 
the minimum set of vertices in a graph G = (V, E) that 
covers all edges. Formally, a subset V’ C V is a ver- 
tex cover if every edge has at least one endpoint in V’. 
Observe that V” is a vertex cover iff V — V” is an inde- 
pendent set. Finding a minimum vertex cover is therefore 
equivalent to finding a maximum independent set. Since 
the latter problem is NP-complete, we conclude that find- 
ing a minimum vertex cover is also NP-complete. Here is 
a straightforward algorithm that achieves approximation 
ratio o(n) = 2, for all n = |V]. 


Vi=6;, HE; 
while E’ Æ do 

select an arbitrary edge wv in F’; 

add u and v to V’; 

remove all edges incident to u or v from E’ 
endwhile. 


Clearly, V’ is a vertex cover. Using adjacency lists with 
links between the two copies of an edge, the running time 
is O(n + m), where m is the number of edges. Further- 
more, we have ọ = 2 because every cover must pick at 
least one vertex of each edge wv selected by the algorithm, 
hence C < 2C*. Observe that this result does not imply 
a constant approximation ratio for the maximum indepen- 
dent set problem. We have |V — V’| = n- © > n—2C%, 
which we have to compare with n — C%, the size of the 
maximum independent set. For C* = 5, the approxima- 
tion ratio is unbounded. 


Let us contemplate the argument we used to relate C' 
and C*. The set of edges uv selected by the algorithm is 
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a matching, that is, a subset of the edges so that no two 
share a vertex. The size of the minimum vertex cover is 
at least the size of the largest possible matching. The al- 
gorithm finds a matching and since it picks two vertices 
per edge, we are guaranteed at most twice as many ver- 
tices as needed. This pattern of bounding C* by the size 
of another quantity (in this case the size of the largest 
matching) is common in the analysis of approximation al- 
gorithms. Incidentally, for bipartite graphs, the size of the 
largest matching is equal to the size of the smallest vertex 
cover. Furthermore, there is a polynomial-time algorithm 
for computing them. 


Traveling salesman. Second, we consider the traveling 
salesman problem, which is formulated for a complete 
graph G = (V, E) with a positive integer cost function 
c: E — Z. A tour in this graph is a Hamiltonian 
cycle and the problem is finding the tour, A, with mini- 
mum total cost, c(A) = J „wea c(uv). Let us first as- 
sume that the cost function satisfies the triangle inequal- 
ity, c(uw) < c(wv) + c(vw) for all u,v,w € V. It can 
be shown that the problem of finding the shortest tour 
remains NP-complete even if we restrict it to weighted 
graphs that satisfy this inequality. We formulate an al- 
gorithm based on the observation that the cost of every 
tour is at least the cost of the minimum spanning tree, 
C* > c(T). 


1 Construct the minimum spanning tree T of G. 


2 Return the preorder sequence of vertices in T. 


Using Prim’s algorithm for the minimum spanning tree, 
the running time is O(n”). Figure 114 illustrates the algo- 
rithm. The preorder sequence is only defined if we have 


Figure 114: The solid minimum spanning tree, the dotted traver- 
sal using each edge of the tree twice, and the solid tour obtained 
by taking short-cuts. 


a root and the neighbors of each vertex are ordered, but 


we may choose both arbitrarily. The cost of the returned 
tour is at most twice the cost of the minimum spanning 
tree. To see this, consider traversing each edge of the min- 
imum spanning tree twice, once in each direction. When- 
ever a vertex is visited more than once, we take the direct 
edge connecting the two neighbors of the second copy as a 
short-cut. By the triangle inequality, this substitution can 
only decrease the overall cost of the traversal. It follows 
that C < 2c(T) < 2C%. 


The triangle inequality is essential in finding a constant 
approximation. Indeed, without it we can construct in- 
stances of the problem for which finding a constant ap- 
proximation is NP-hard. To see this, transform an un- 
weighted graph G” = (V’, E’) to the complete weighted 
graph G = (V, E) with 


ww) = li if uv € BY, 
< > on+1 otherwise. 


Any @-approximation algorithm must return the Hamilto- 
nian cycle of G”, if there is one. 


Set cover. Third, we consider the problem of covering 
a set X with sets chosen from a set system F. We as- 
sume the set is the union of sets in the system, X = UF. 
More precisely, we are looking for a smallest subsystem 
F' C F with X = UF". The cost of this subsystem is 
the number of sets it contains, |F'|. See Figure 115 for 
an illustration of the problem. The vertex cover problem 


-JU 


Figure 115: The set X of twelve dots can be covered with four 
of the five sets in the system. 


is a special case: X = E and F contains all subsets of 
edges incident to a common vertex. Itis special because 
each element (edge) belongs to exactly two sets. Since we 
no longer have a bound on the number of sets containing 
a single element, it is not surprising that the algorithm for 
vertex covers does not extend to a constant-approximation 
algorithm for set covers. Instead, we consider the follow- 
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ing greedy approach that selects, at each step, the set con- 
taining the maximum number of yet uncovered elements. 


Fli= 0; X'=X; 
while X’ # do 
select S € F maximizing |S N X’|; 
Fi =F'U{S}; X'= X'-S 
endwhile. 


Using a sparse matrix representation of the set system 
(similar to an adjacency list representation of a graph), we 
can run the algorithm in time proportional to the total size 
of the sets in the system, n = ` gep |S]. We omit the 
details. 


Analysis. More interesting than the running time is the 
analysis of the approximation ratio the greedy algorithm 
achieves. It is convenient to have short notation for the d- 
th harmonic number, Ha = £L 4 for d > 0. Recall that 
Ag <1+I1nd for d > 1. Let the size of the largest set in 
the system be m = max{|$| | S € F}. 


CLAIM. The greedy method is an Hm-approximation al- 
gorithm for the set cover problem. 


PROOF. For each set S selected by the algorithm, we dis- 
tribute $1 over the |S N X’| elements covered for the first 
time. Let cy be the cost allocated this way to x € X. We 
have |F"| = )0.cx cr. If x is covered the first time by the 
i-th selected set, S;, then 


1 
|S; — ($1 U... U Si-1)| 


Cy = 


We have |F"| < Ð ser- } res Cx because the optimal 
cover, F*, contains each element x at least once. We will 
prove shortly that X` eg €s < Hyg) for every set S € F. 
It follows that 

IFI < $, Als) < HmlF"*l, 
SEF* 


For m = 3, we get 90 = H3 = H, This implies that 
for graphs with vertex-degrees at most 3, the greedy algo- 
rithm guarantees a vertex cover of size at most 2 times 
the optimum, which is better than the ratio 2 guaranteed 


by our first algorithm. 


as claimed. 


We still need to prove that the sum of costs Cy over the 
elements of a set S in the system is bounded from above 
by Hj). Let u; be the number of elements in S that are 


not covered by the first i selected sets, u; = |S — (S1 U 
...U S;)|, and observe that the numbers do not increase. 
Let u,—1 be the last non-zero number in the sequence, so 
|S| = uo >... > Up-1 > Up = 0. Since uj_1 — u; is the 
number of elements in S covered the first time by S;, we 
have 


k 
Ui-1 — Ui 
Jue = 2 GUUS 


TES 


We also have u;—1 < |S; — (S1 U ... U Si—1)|, for all 
i < k, because of the greedy choice of S;. If this were 
not the case, the algorithm would have chosen S instead 
of S; in the construction of F’. The problem thus reduces 
to bounding the sum of ratios “—**._ It is not difficult 
to see that this sum can be at least logarithmic in the size 
of S. Indeed, if we choose u; about half the size of u;_1, 
for all ¿ > 1, then we have logarithmically many terms, 
each roughly 4. We use a sequence of simple arithmetic 
manipulations to prove that this lower bound is asymptot- 
ically tight: 


vi u 
i—1 — Uj 
> Ce < > — 
Uj 
zES i=l wl 
k Ui-1 
i=l joujt1 mI 


We now replace the denominator by j < ui—1 to form a 
telescoping series of harmonic numbers and get 


ae 


M 
M 


zES j=j jou; +1 
i= faa ait 
k 
= XN Hua Hau) 
i= 


This is equal to Hus — Hu, = Hs, which fills the gap 
left in the analysis of the greedy algorithm. 
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Seventh Homework Assignment 


The purpose of this assignment is to help you prepare for 
the final exam. Solutions will neither be graded nor even 
collected. 


Problem 1. (20 = 5 + 15 points). Consider the class 
of satisfiable boolean formulas in conjunctive nor- 
mal form in which each clause contains two literals, 
2-SAT = {y € SAT | is 2-CNF}. 


(a) Is 2-SAT € NP? 

(b) Is there a polynomial-time algorithm for decid- 
ing whether or not a boolean formula in 2-CNF 
is satisfiable? If your answer is yes, then de- 


scribe and analyze your algorithm. If your an- 
swer is no, then show that 2-SAT € NPC. 


Problem 2. (20 points). Let A be a finite set and f a func- 
tion that maps every a € A to a positive integer f(a). 
The PARTITION problem asks whether or not there is 
a subset B C A such that 


Sie = +, fo. 


beB ac€A-B 


We have learned that the PARTITION problem is 
NP-complete. Given positive integers j and k, the 
SUM OF SQUARES problem asks whether or not 
A can be partitioned into j disjoint subsets, A = 
Bı Ü B2 Ù ... Ù Bj, such that 


D (x so) < k. 


i=1 \aEBi 


Prove that the SUM OF SQUARES problem is NP- 
complete. 


Problem 3. (20 = 10+10 points). Let G be an undirected 
graph. A path in G is simple if it contains each ver- 
tex at most once. Specifying two vertices u,v and a 
positive integer k, the LONGEST PATH problem asks 
whether or not there is a simple path connecting u 
and v whose length is k or longer. 


(a) Give a polynomial-time algorithm for the 
LONGEST PATH problem or show that it is NP- 
hard. 


(b) Revisit (a) under the assumption that G is di- 
rected and acyclic. 
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Problem 4. (20 = 10 + 10 points). Let A C 2” be an 


abstract simplicial complex over the finite set V and 
let k be a positive integer. 


(a) Is it NP-hard to decide whether A has k or more 
disjoint simplices? 

(b) Is it NP-hard to decide whether A has k or 
fewer simplices whose union is V? 


Problem 5. (20 points). Let G = (V,£) be an undi- 


rected, bipartite graph and recall that there is a 
polynomial-time algorithm for constructing a max- 
imum matching. We are interested in computing a 
minimum set of matchings such that every edge of 
the graph is a member of at least one of the selected 
matchings. Give a polynomial-time algorithm con- 
structing an O(log) approximation for this prob- 
lem. 


